[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:52:39 dist-test-slave-ws74 krb5kdc[774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:52:39 dist-test-slave-ws74 krb5kdc[774](info): set up 2 sockets
Sep 26 06:52:39 dist-test-slave-ws74 krb5kdc[774](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 06:52:40.502270 767 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:52:40 dist-test-slave-ws74 krb5kdc[774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:52:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:52:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:52:40.615877 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:34937
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:34937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:40.693308 787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:40.693508 787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:40.693529 787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:40.694607 787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:52:40.694653 787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:40.694666 787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:40.694711 787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:52:40.694729 787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:52:40.696066 787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:34937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:34937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:40.696233 787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:40.696537 787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:52:40.701771 787 server_base.cc:1034] running on GCE node
W20260926 06:52:40.701813 793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:40.701910 795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:40.701813 792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:40.702217 787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:40.702486 787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:40.703650 787 hybrid_clock.cc:648] HybridClock initialized: now 1790405560703634 us; error 28 us; skew 500 ppm
Sep 26 06:52:40 dist-test-slave-ws74 krb5kdc[774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405560, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:40.705972 787 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:52:40.706456 787 webserver.cc:460] Webserver started at http://127.0.191.254:35531/ using document root <none> and password file <none>
I20260926 06:52:40.706621 787 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:40.706660 787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:40.706738 787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:40.707674 787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "1563f2768b96464ab552f9d1935acd5e"
format_stamp: "Formatted at 2026-09-26 06:52:40 on dist-test-slave-ws74"
server_key: "68026949e8d30cf658ba0703cea95898"
server_key_iv: "caec1ff3d71e81125da46ccce7c500d1"
server_key_version: "encryptionkey@0"
I20260926 06:52:40.707964 787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "1563f2768b96464ab552f9d1935acd5e"
format_stamp: "Formatted at 2026-09-26 06:52:40 on dist-test-slave-ws74"
server_key: "68026949e8d30cf658ba0703cea95898"
server_key_iv: "caec1ff3d71e81125da46ccce7c500d1"
server_key_version: "encryptionkey@0"
I20260926 06:52:40.709198 787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:52:40.709892 802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:40.710049 787 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:40.710131 787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "1563f2768b96464ab552f9d1935acd5e"
format_stamp: "Formatted at 2026-09-26 06:52:40 on dist-test-slave-ws74"
server_key: "68026949e8d30cf658ba0703cea95898"
server_key_iv: "caec1ff3d71e81125da46ccce7c500d1"
server_key_version: "encryptionkey@0"
I20260926 06:52:40.710198 787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:40.744433 787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:40.747013 787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:40.747189 787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:40.751327 787 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:34937
I20260926 06:52:40.751358 854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:34937 every 8 connection(s)
I20260926 06:52:40.751694 787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:52:40.752493 855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:40.754670 855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e: Bootstrap starting.
I20260926 06:52:40.755187 855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:40.755407 855 log.cc:824] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:40.756052 855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e: No bootstrap required, opened a new log
I20260926 06:52:40.757624 855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1563f2768b96464ab552f9d1935acd5e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 34937 } }
I20260926 06:52:40.757753 855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:40.757776 855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1563f2768b96464ab552f9d1935acd5e, State: Initialized, Role: FOLLOWER
I20260926 06:52:40.757959 855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1563f2768b96464ab552f9d1935acd5e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 34937 } }
I20260926 06:52:40.758038 855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:52:40.758059 855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:52:40.758087 855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:40.758538 855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1563f2768b96464ab552f9d1935acd5e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 34937 } }
I20260926 06:52:40.758615 855 leader_election.cc:302] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1563f2768b96464ab552f9d1935acd5e; no voters:
I20260926 06:52:40.758764 855 leader_election.cc:288] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:52:40.758785 858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:40.758896 858 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [term 1 LEADER]: Becoming Leader. State: Replica: 1563f2768b96464ab552f9d1935acd5e, State: Running, Role: LEADER
I20260926 06:52:40.759011 855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:52:40.759060 858 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1563f2768b96464ab552f9d1935acd5e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 34937 } }
I20260926 06:52:40.759585 859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1563f2768b96464ab552f9d1935acd5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1563f2768b96464ab552f9d1935acd5e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 34937 } } }
I20260926 06:52:40.759680 859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:40.759703 860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1563f2768b96464ab552f9d1935acd5e. Latest consensus state: current_term: 1 leader_uuid: "1563f2768b96464ab552f9d1935acd5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1563f2768b96464ab552f9d1935acd5e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 34937 } } }
I20260926 06:52:40.759862 860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:40.760056 867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:52:40.760236 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 787
I20260926 06:52:40.760344 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:52:40.760550 867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:52:40.761049 767 external_mini_cluster.cc:1428] Setting key 42284363c2f926dc72902d29e48372b2
I20260926 06:52:40.762224 867 catalog_manager.cc:1347] Generated new cluster ID: e113d570543943a79539f19f8cf1c0e0
I20260926 06:52:40.762276 867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:52:40.766669 867 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:52:40.767213 867 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:52:40.772383 867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1563f2768b96464ab552f9d1935acd5e: Generated new TSK 0
I20260926 06:52:40.772541 867 catalog_manager.cc:1514] Initializing in-progress tserver states...
Sep 26 06:52:40 dist-test-slave-ws74 krb5kdc[774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:40.777913 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:40.761916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55284 (local address 127.0.191.254:34937)
0926 06:52:40.762063 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:40.762072 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:40.762444 (+ 372us) server_negotiation.cc:407] Connection header received
0926 06:52:40.764457 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:40.764464 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:40.764515 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:40.764664 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:40.766571 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:40.768389 (+ 1818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:40.769810 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:40.770402 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:40.770508 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:40.774321 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:40.774326 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:40.774326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:40.774336 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:40.776163 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:40.776547 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:40.776550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:40.776550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:40.776660 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:40.776894 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:40.776895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:40.776895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:40.777155 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:40.777306 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:40.777559 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 06:52:40.777716 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:52:40.808053 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:34937
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:40.887853 881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:40.888046 881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:40.888068 881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:40.889204 881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:40.889245 881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:40.889297 881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:52:40.890723 881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:34937
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:40.890955 881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:40.891194 881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:40.896198 886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:40.896238 881 server_base.cc:1034] running on GCE node
W20260926 06:52:40.896209 889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:40.896225 887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:40.896654 881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:40.896842 881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:40.898010 881 hybrid_clock.cc:648] HybridClock initialized: now 1790405560897999 us; error 27 us; skew 500 ppm
Sep 26 06:52:40 dist-test-slave-ws74 krb5kdc[774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405560, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:40.900460 881 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:52:40.900877 881 webserver.cc:460] Webserver started at http://127.0.191.193:42505/ using document root <none> and password file <none>
I20260926 06:52:40.901108 881 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:40.901175 881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:40.901328 881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:40.902411 881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "38328a2c8daf49ce8c5645893b5be685"
format_stamp: "Formatted at 2026-09-26 06:52:40 on dist-test-slave-ws74"
server_key: "7921239927f40d9ee46976a0bc76e2ec"
server_key_iv: "1dea636f9a364e25a379b927daf8516a"
server_key_version: "encryptionkey@0"
I20260926 06:52:40.902773 881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "38328a2c8daf49ce8c5645893b5be685"
format_stamp: "Formatted at 2026-09-26 06:52:40 on dist-test-slave-ws74"
server_key: "7921239927f40d9ee46976a0bc76e2ec"
server_key_iv: "1dea636f9a364e25a379b927daf8516a"
server_key_version: "encryptionkey@0"
I20260926 06:52:40.904435 881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:52:40.905212 896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:40.905428 881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:52:40.905532 881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "38328a2c8daf49ce8c5645893b5be685"
format_stamp: "Formatted at 2026-09-26 06:52:40 on dist-test-slave-ws74"
server_key: "7921239927f40d9ee46976a0bc76e2ec"
server_key_iv: "1dea636f9a364e25a379b927daf8516a"
server_key_version: "encryptionkey@0"
I20260926 06:52:40.905632 881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:40.932858 881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:40.935297 881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:40.935465 881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:40.935760 881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:52:40.936084 881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:40.936142 881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:40.936187 881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:40.936214 881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:40.942669 881 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:43919
I20260926 06:52:40.942710 1009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:43919 every 8 connection(s)
I20260926 06:52:40.943101 881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:52:40 dist-test-slave-ws74 krb5kdc[774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405560, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:40.952099 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:40.943945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:35469 (local address 127.0.191.254:34937)
0926 06:52:40.944087 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:40.944088 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:40.944224 (+ 136us) server_negotiation.cc:407] Connection header received
0926 06:52:40.944699 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:40.944700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:40.944710 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:40.944772 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:40.945809 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:40.946329 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:40.947383 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:40.947570 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:40.947618 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:40.949770 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:40.949771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:40.949771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:40.949777 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:40.950914 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:40.951330 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:40.951331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:40.951331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:40.951380 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:40.951600 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:40.951600 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:40.951600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:40.951691 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:40.951740 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:40.951974 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 06:52:40.952038 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260926 06:52:40.952325 1012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:40.943915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:34937 (local address 127.0.191.193:35469)
0926 06:52:40.944074 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:40.944118 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:40.944610 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:40.944823 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:40.944826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:40.945480 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:40.945704 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:40.945709 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:40.946419 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:40.946420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:40.947306 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:40.947308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:40.947642 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:40.947643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:40.947750 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:40.948298 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:40.948307 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:40.949690 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:40.950990 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:40.950996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:40.950997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:40.951263 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:40.951441 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:40.951441 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:40.951442 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:40.951545 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:40.951747 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:40.951751 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:40.951861 (+ 110us) client_negotiation.cc:769] Sending connection context
0926 06:52:40.951946 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 06:52:40.952096 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260926 06:52:40.952867 1010 heartbeater.cc:347] Connected to a master server at 127.0.191.254:34937
I20260926 06:52:40.952972 1010 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:40.953017 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 881
I20260926 06:52:40.953111 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:52:40.953210 1010 heartbeater.cc:511] Master 127.0.191.254:34937 requested a full tablet report, sending...
I20260926 06:52:40.953308 767 external_mini_cluster.cc:1428] Setting key 530b09b30dde27b4ce435c8a965cc8c6
I20260926 06:52:40.953791 819 ts_manager.cc:194] Registered new tserver with Master: 38328a2c8daf49ce8c5645893b5be685 (127.0.191.193:43919)
I20260926 06:52:40.954706 819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:35469
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:52:40.978444 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:34937
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:41.057315 1015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:41.057561 1015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:41.057602 1015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:41.058750 1015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:41.058817 1015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:41.058864 1015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:52:41.060451 1015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:34937
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:41.060725 1015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:41.060938 1015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:41.065973 1020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:41.066032 1023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:41.065981 1021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:41.066260 1015 server_base.cc:1034] running on GCE node
I20260926 06:52:41.066483 1015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:41.066747 1015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:41.067956 1015 hybrid_clock.cc:648] HybridClock initialized: now 1790405561067942 us; error 37 us; skew 500 ppm
Sep 26 06:52:41 dist-test-slave-ws74 krb5kdc[774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405561, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:41.071295 1015 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:52:41.071802 1015 webserver.cc:460] Webserver started at http://127.0.191.194:39313/ using document root <none> and password file <none>
I20260926 06:52:41.072089 1015 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:41.072190 1015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:41.072340 1015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:41.073601 1015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "45c3a524220d4a1090331b2a072a8abd"
format_stamp: "Formatted at 2026-09-26 06:52:41 on dist-test-slave-ws74"
server_key: "1c4dfe313f0ee1cc541d4d68c836876a"
server_key_iv: "0719b8a21895d707aead2a763f0fc032"
server_key_version: "encryptionkey@0"
I20260926 06:52:41.074030 1015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "45c3a524220d4a1090331b2a072a8abd"
format_stamp: "Formatted at 2026-09-26 06:52:41 on dist-test-slave-ws74"
server_key: "1c4dfe313f0ee1cc541d4d68c836876a"
server_key_iv: "0719b8a21895d707aead2a763f0fc032"
server_key_version: "encryptionkey@0"
I20260926 06:52:41.075843 1015 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 06:52:41.076682 1030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:41.076857 1015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:52:41.076948 1015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "45c3a524220d4a1090331b2a072a8abd"
format_stamp: "Formatted at 2026-09-26 06:52:41 on dist-test-slave-ws74"
server_key: "1c4dfe313f0ee1cc541d4d68c836876a"
server_key_iv: "0719b8a21895d707aead2a763f0fc032"
server_key_version: "encryptionkey@0"
I20260926 06:52:41.077014 1015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:41.085021 1015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:41.086726 1015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:41.086867 1015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:41.087082 1015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:52:41.087368 1015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:41.087399 1015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:41.087417 1015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:41.087462 1015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:41.093851 1015 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:46251
I20260926 06:52:41.093937 1143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:46251 every 8 connection(s)
I20260926 06:52:41.094327 1015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:52:41 dist-test-slave-ws74 krb5kdc[774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405561, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:41.102900 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 1015
I20260926 06:52:41.103011 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:52:41.103190 767 external_mini_cluster.cc:1428] Setting key 3667d41b1524cbe67e376742e21cad40
I20260926 06:52:41.103304 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:41.095202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:42071 (local address 127.0.191.254:34937)
0926 06:52:41.095273 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:41.095276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:41.095459 (+ 183us) server_negotiation.cc:407] Connection header received
0926 06:52:41.096132 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:41.096134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:41.096149 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:41.096219 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:41.097062 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:41.097635 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:41.098651 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:41.098826 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:41.098874 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:41.100751 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:41.100752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:41.100752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:41.100758 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:41.101989 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:41.102345 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:41.102345 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:41.102345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:41.102393 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:41.102587 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:41.102588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:41.102588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:41.102680 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:41.102728 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:41.103107 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 06:52:41.103194 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 06:52:41.103817 1146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:41.095167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:34937 (local address 127.0.191.194:42071)
0926 06:52:41.095341 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:41.095385 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:41.096010 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:41.096271 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:41.096274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:41.096795 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:41.096985 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:41.096989 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:41.097714 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:41.097715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:41.098578 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:41.098580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:41.098893 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:41.098893 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:41.098985 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:41.099408 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:41.099415 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:41.100670 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:41.102067 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:41.102072 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:41.102074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:41.102282 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:41.102449 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:41.102449 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:41.102449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:41.102536 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:41.102733 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:41.102754 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:41.102912 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 06:52:41.103427 (+ 515us) client_negotiation.cc:240] Negotiation successful
0926 06:52:41.103593 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260926 06:52:41.104266 1144 heartbeater.cc:347] Connected to a master server at 127.0.191.254:34937
I20260926 06:52:41.104358 1144 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:41.104538 1144 heartbeater.cc:511] Master 127.0.191.254:34937 requested a full tablet report, sending...
I20260926 06:52:41.104885 819 ts_manager.cc:194] Registered new tserver with Master: 45c3a524220d4a1090331b2a072a8abd (127.0.191.194:46251)
I20260926 06:52:41.105257 819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:42071
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:52:41.128052 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:34937
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:41.206290 1149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:41.206506 1149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:41.206543 1149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:41.207703 1149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:41.207768 1149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:41.207813 1149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:52:41.209261 1149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:34937
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:41.209532 1149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:41.209772 1149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:41.214501 1155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:41.214490 1154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:41.214576 1157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:41.214648 1149 server_base.cc:1034] running on GCE node
I20260926 06:52:41.215276 1149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:41.215498 1149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:41.216650 1149 hybrid_clock.cc:648] HybridClock initialized: now 1790405561216639 us; error 42 us; skew 500 ppm
Sep 26 06:52:41 dist-test-slave-ws74 krb5kdc[774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405561, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:41.219031 1149 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:52:41.219434 1149 webserver.cc:460] Webserver started at http://127.0.191.195:46177/ using document root <none> and password file <none>
I20260926 06:52:41.219658 1149 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:41.219722 1149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:41.219871 1149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:41.220830 1149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "b1cf883356694d59ae4662509925bd69"
format_stamp: "Formatted at 2026-09-26 06:52:41 on dist-test-slave-ws74"
server_key: "12ca8afb3a8924cf474019d92bb76dfb"
server_key_iv: "6ac1ffc277cba880687389adaedb4698"
server_key_version: "encryptionkey@0"
I20260926 06:52:41.221170 1149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "b1cf883356694d59ae4662509925bd69"
format_stamp: "Formatted at 2026-09-26 06:52:41 on dist-test-slave-ws74"
server_key: "12ca8afb3a8924cf474019d92bb76dfb"
server_key_iv: "6ac1ffc277cba880687389adaedb4698"
server_key_version: "encryptionkey@0"
I20260926 06:52:41.222599 1149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:52:41.223290 1164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:41.223460 1149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:52:41.223551 1149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "b1cf883356694d59ae4662509925bd69"
format_stamp: "Formatted at 2026-09-26 06:52:41 on dist-test-slave-ws74"
server_key: "12ca8afb3a8924cf474019d92bb76dfb"
server_key_iv: "6ac1ffc277cba880687389adaedb4698"
server_key_version: "encryptionkey@0"
I20260926 06:52:41.223634 1149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:41.232543 1149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:41.234411 1149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:41.234558 1149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:41.235136 1149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:52:41.235419 1149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:41.235471 1149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:41.235508 1149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:41.235533 1149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:41.241668 1149 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:41999
I20260926 06:52:41.241717 1277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:41999 every 8 connection(s)
I20260926 06:52:41.242007 1149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:52:41 dist-test-slave-ws74 krb5kdc[774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405561, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:41.250197 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:41.242674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:59831 (local address 127.0.191.254:34937)
0926 06:52:41.242754 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:41.242756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:41.242929 (+ 173us) server_negotiation.cc:407] Connection header received
0926 06:52:41.243365 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:41.243365 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:41.243375 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:41.243436 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:41.244261 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:41.244761 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:41.245822 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:41.246003 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:41.246055 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:41.247969 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:41.247970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:41.247971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:41.247977 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:41.249147 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:41.249498 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:41.249499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:41.249499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:41.249548 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:41.249746 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:41.249747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:41.249747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:41.249842 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:41.249891 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:41.250084 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 06:52:41.250138 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 06:52:41.250442 1280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:41.242622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:34937 (local address 127.0.191.195:59831)
0926 06:52:41.242814 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:41.242858 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:41.243288 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:41.243487 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:41.243490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:41.244006 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:41.244181 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:41.244185 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:41.244836 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:41.244837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:41.245747 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:41.245749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:41.246071 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:41.246072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:41.246161 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:41.246563 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:41.246570 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:41.247889 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:41.249223 (+ 1334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:41.249228 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:41.249229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:41.249449 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:41.249605 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:41.249605 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:41.249605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:41.249690 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:41.249895 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:41.249898 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:41.249997 (+ 99us) client_negotiation.cc:769] Sending connection context
0926 06:52:41.250072 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:52:41.250180 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":87,"threads_started":1}
I20260926 06:52:41.251044 1278 heartbeater.cc:347] Connected to a master server at 127.0.191.254:34937
I20260926 06:52:41.251132 1278 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:41.251266 1278 heartbeater.cc:511] Master 127.0.191.254:34937 requested a full tablet report, sending...
I20260926 06:52:41.251614 819 ts_manager.cc:194] Registered new tserver with Master: b1cf883356694d59ae4662509925bd69 (127.0.191.195:41999)
I20260926 06:52:41.252043 819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:59831
I20260926 06:52:41.252128 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 1149
I20260926 06:52:41.252216 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:52:41.252393 767 external_mini_cluster.cc:1428] Setting key 38e0a0d110a30ee56d6a33f3019d47d1
I20260926 06:52:41.253432 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 06:52:41 dist-test-slave-ws74 krb5kdc[774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405561, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 06:52:41 dist-test-slave-ws74 krb5kdc[774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405561, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.191.193@KRBTEST.COM
I20260926 06:52:41.275312 1285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:41.268113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59452 (local address 127.0.191.193:43919)
0926 06:52:41.268269 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:41.268273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:41.268282 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:52:41.268324 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:41.268326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:41.268338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:41.268399 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:41.268903 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:41.269658 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:41.270356 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:41.270665 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:41.270732 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:41.273052 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:41.273058 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:41.273059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:41.273065 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:41.274415 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:41.274701 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:41.274703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:41.274704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:41.274766 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:41.274902 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:41.274903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:41.274903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:41.275004 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:41.275098 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:41.275150 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 06:52:41.275205 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
W20260926 06:52:41.275646 924 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59452
Sep 26 06:52:41 dist-test-slave-ws74 krb5kdc[774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 06:52:41 dist-test-slave-ws74 krb5kdc[774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.193@KRBTEST.COM
I20260926 06:52:41.296572 1285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:41.290135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59456 (local address 127.0.191.193:43919)
0926 06:52:41.290219 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:41.290221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:41.290294 (+ 73us) server_negotiation.cc:407] Connection header received
0926 06:52:41.290386 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:41.290387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:41.290398 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:41.290470 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:41.291006 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:41.291483 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:41.292236 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:41.292439 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:41.292512 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:41.294583 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:41.294584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:41.294584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:41.294592 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:41.295702 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:41.295999 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:41.295999 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:41.296000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:41.296045 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:41.296199 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:41.296199 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:41.296200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:41.296286 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:41.296369 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:41.296445 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 06:52:41.296513 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 06:52:41.297052 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 881
I20260926 06:52:41.301304 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 1015
I20260926 06:52:41.306890 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 1149
I20260926 06:52:41.311245 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 787
2026-09-26T06:52:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1839 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:52:41 dist-test-slave-ws74 krb5kdc[1292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:52:41 dist-test-slave-ws74 krb5kdc[1292](info): set up 2 sockets
Sep 26 06:52:41 dist-test-slave-ws74 krb5kdc[1292](info): commencing operation
krb5kdc: starting...
W20260926 06:52:42.364982 767 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:52:42 dist-test-slave-ws74 krb5kdc[1292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:52:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:52:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:52:42.484644 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33429
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:33429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:42.563977 1305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:42.564167 1305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:42.564188 1305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:42.565330 1305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:52:42.565409 1305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:42.565443 1305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:42.565462 1305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:52:42.565475 1305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:52:42.566835 1305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:33429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:42.567011 1305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:42.567162 1305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:42.571848 1310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:42.571853 1311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:42.572029 1305 server_base.cc:1034] running on GCE node
W20260926 06:52:42.571897 1313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:42.572329 1305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:42.572597 1305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:42.573764 1305 hybrid_clock.cc:648] HybridClock initialized: now 1790405562573761 us; error 39 us; skew 500 ppm
Sep 26 06:52:42 dist-test-slave-ws74 krb5kdc[1292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405562, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:42.580754 1305 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:52:42.581216 1305 webserver.cc:460] Webserver started at http://127.0.191.254:34081/ using document root <none> and password file <none>
I20260926 06:52:42.581494 1305 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:42.581542 1305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:42.581758 1305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:42.582813 1305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "796e420f4bfe47a6914d4f1613b38d65"
format_stamp: "Formatted at 2026-09-26 06:52:42 on dist-test-slave-ws74"
server_key: "55256c9a9da28e5f276b9155c2439f79"
server_key_iv: "3e2bfb1a04541f3bc6e40fbfa7df7a22"
server_key_version: "encryptionkey@0"
I20260926 06:52:42.583206 1305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "796e420f4bfe47a6914d4f1613b38d65"
format_stamp: "Formatted at 2026-09-26 06:52:42 on dist-test-slave-ws74"
server_key: "55256c9a9da28e5f276b9155c2439f79"
server_key_iv: "3e2bfb1a04541f3bc6e40fbfa7df7a22"
server_key_version: "encryptionkey@0"
I20260926 06:52:42.584482 1305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:52:42.585171 1320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:42.585363 1305 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:52:42.585501 1305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "796e420f4bfe47a6914d4f1613b38d65"
format_stamp: "Formatted at 2026-09-26 06:52:42 on dist-test-slave-ws74"
server_key: "55256c9a9da28e5f276b9155c2439f79"
server_key_iv: "3e2bfb1a04541f3bc6e40fbfa7df7a22"
server_key_version: "encryptionkey@0"
I20260926 06:52:42.585569 1305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:42.602919 1305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:42.605173 1305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:42.605283 1305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:42.609414 1305 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:33429
I20260926 06:52:42.609455 1372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:33429 every 8 connection(s)
I20260926 06:52:42.609822 1305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:52:42.610613 1373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:42.612874 1373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65: Bootstrap starting.
I20260926 06:52:42.613480 1373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:42.613767 1373 log.cc:824] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:42.614426 1373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65: No bootstrap required, opened a new log
I20260926 06:52:42.616009 1373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796e420f4bfe47a6914d4f1613b38d65" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33429 } }
I20260926 06:52:42.616137 1373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:42.616182 1373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 796e420f4bfe47a6914d4f1613b38d65, State: Initialized, Role: FOLLOWER
I20260926 06:52:42.616334 1373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796e420f4bfe47a6914d4f1613b38d65" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33429 } }
I20260926 06:52:42.616391 1373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:52:42.616432 1373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:52:42.616478 1373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:42.616950 1373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796e420f4bfe47a6914d4f1613b38d65" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33429 } }
I20260926 06:52:42.617056 1373 leader_election.cc:302] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 796e420f4bfe47a6914d4f1613b38d65; no voters:
I20260926 06:52:42.617237 1373 leader_election.cc:288] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:52:42.617277 1376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:42.617498 1373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:52:42.617763 1376 raft_consensus.cc:686] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [term 1 LEADER]: Becoming Leader. State: Replica: 796e420f4bfe47a6914d4f1613b38d65, State: Running, Role: LEADER
I20260926 06:52:42.617851 1376 consensus_queue.cc:237] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796e420f4bfe47a6914d4f1613b38d65" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33429 } }
I20260926 06:52:42.618198 1377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "796e420f4bfe47a6914d4f1613b38d65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796e420f4bfe47a6914d4f1613b38d65" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33429 } } }
I20260926 06:52:42.618214 1378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 796e420f4bfe47a6914d4f1613b38d65. Latest consensus state: current_term: 1 leader_uuid: "796e420f4bfe47a6914d4f1613b38d65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796e420f4bfe47a6914d4f1613b38d65" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33429 } } }
I20260926 06:52:42.618279 1377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:42.618306 1378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65 [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:42.618481 1385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:52:42.619149 1385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:52:42.619367 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 1305
I20260926 06:52:42.619451 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:52:42.619612 767 external_mini_cluster.cc:1428] Setting key 7f0f46b0b788a4750d41bb7fe869b553
I20260926 06:52:42.620731 1385 catalog_manager.cc:1347] Generated new cluster ID: bca58bed7413454397d4b72f0bd45aa9
I20260926 06:52:42.620785 1385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:52:42 dist-test-slave-ws74 krb5kdc[1292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:42.631763 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:42.620058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49826 (local address 127.0.191.254:33429)
0926 06:52:42.620371 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:42.620380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:42.620413 (+ 33us) server_negotiation.cc:407] Connection header received
0926 06:52:42.620524 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:42.620531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:42.620594 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:42.621490 (+ 896us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:42.622193 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:42.623614 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:42.624683 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:42.625292 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:42.625416 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:42.628138 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:42.628141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:42.628142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:42.628153 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:42.630078 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:42.630493 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:42.630495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:42.630496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:42.630593 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:42.630797 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:42.630798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:42.630798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:42.631080 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:42.631211 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:42.631389 (+ 178us) server_negotiation.cc:299] Negotiation successful
0926 06:52:42.631521 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260926 06:52:42.633399 1385 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:52:42.633867 1385 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:52:42.639705 1385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 796e420f4bfe47a6914d4f1613b38d65: Generated new TSK 0
I20260926 06:52:42.639883 1385 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:52:42.664187 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33429
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:42.743811 1399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:42.744051 1399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:42.744093 1399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:42.745278 1399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:42.745347 1399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:42.745416 1399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:52:42.746871 1399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33429
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:42.747138 1399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:42.747370 1399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:52:42.752269 1399 server_base.cc:1034] running on GCE node
W20260926 06:52:42.752207 1405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:42.752219 1407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:42.752215 1404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:42.752640 1399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:42.752907 1399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:42.754068 1399 hybrid_clock.cc:648] HybridClock initialized: now 1790405562754057 us; error 25 us; skew 500 ppm
Sep 26 06:52:42 dist-test-slave-ws74 krb5kdc[1292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405562, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:42.756454 1399 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:52:42.756950 1399 webserver.cc:460] Webserver started at http://127.0.191.193:39875/ using document root <none> and password file <none>
I20260926 06:52:42.757187 1399 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:42.757256 1399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:42.757445 1399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:42.758466 1399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "17933da43d3542e695852213175df76f"
format_stamp: "Formatted at 2026-09-26 06:52:42 on dist-test-slave-ws74"
server_key: "da686d64cc3002d60287ef397602d5b7"
server_key_iv: "62dd7d9c135bf118a09ed981fad1fef5"
server_key_version: "encryptionkey@0"
I20260926 06:52:42.758824 1399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "17933da43d3542e695852213175df76f"
format_stamp: "Formatted at 2026-09-26 06:52:42 on dist-test-slave-ws74"
server_key: "da686d64cc3002d60287ef397602d5b7"
server_key_iv: "62dd7d9c135bf118a09ed981fad1fef5"
server_key_version: "encryptionkey@0"
I20260926 06:52:42.760128 1399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:52:42.760826 1414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:42.760998 1399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:52:42.761093 1399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "17933da43d3542e695852213175df76f"
format_stamp: "Formatted at 2026-09-26 06:52:42 on dist-test-slave-ws74"
server_key: "da686d64cc3002d60287ef397602d5b7"
server_key_iv: "62dd7d9c135bf118a09ed981fad1fef5"
server_key_version: "encryptionkey@0"
I20260926 06:52:42.761179 1399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:42.778795 1399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:42.780576 1399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:42.780745 1399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:42.780978 1399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:52:42.781330 1399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:42.781410 1399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:42.781455 1399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:42.781477 1399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:42.787305 1399 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:45037
I20260926 06:52:42.787348 1527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:45037 every 8 connection(s)
I20260926 06:52:42.787670 1399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:52:42.788721 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 1399
I20260926 06:52:42.788815 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:52:42.788966 767 external_mini_cluster.cc:1428] Setting key f042474ee61a28fc28adc5135c28ff9d
Sep 26 06:52:42 dist-test-slave-ws74 krb5kdc[1292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405562, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
I20260926 06:52:42.799009 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:42.788361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:59937 (local address 127.0.191.254:33429)
0926 06:52:42.788427 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:42.788430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:42.788661 (+ 231us) server_negotiation.cc:407] Connection header received
0926 06:52:42.789515 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:42.789517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:42.789531 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:42.789584 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:42.791029 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:42.791841 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:42.793375 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:42.793631 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:42.793684 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:42.796269 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:42.796271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:42.796271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:42.796277 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:42.797807 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:42.798181 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:42.798182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:42.798182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:42.798236 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:42.798406 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:42.798407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:42.798407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:42.798547 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:42.798600 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:42.798878 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 06:52:42.798944 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 06:52:42.799181 1530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:42.788354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33429 (local address 127.0.191.193:59937)
0926 06:52:42.788509 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:42.788563 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:42.789342 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:42.789712 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:42.789716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:42.790619 (+ 903us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:42.790914 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:42.790920 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:42.791935 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:42.791936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:42.793259 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:42.793262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:42.793725 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:42.793726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:42.793886 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:42.794416 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:42.794427 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:42.796158 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:42.797882 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:42.797889 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:42.797892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:42.798113 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:42.798276 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:42.798276 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:42.798276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:42.798358 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:42.798597 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:42.798602 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:42.798759 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 06:52:42.798851 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 06:52:42.798971 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260926 06:52:42.799701 1528 heartbeater.cc:347] Connected to a master server at 127.0.191.254:33429
I20260926 06:52:42.799823 1528 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:42.800015 1528 heartbeater.cc:511] Master 127.0.191.254:33429 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
I20260926 06:52:42.800559 1337 ts_manager.cc:194] Registered new tserver with Master: 17933da43d3542e695852213175df76f (127.0.191.193:45037)
I20260926 06:52:42.801152 1337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:59937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:52:42.816542 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33429
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:42.897735 1533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:42.897928 1533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:42.897950 1533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:42.899016 1533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:42.899049 1533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:42.899075 1533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:52:42.900388 1533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33429
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:42.900553 1533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:42.900787 1533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:42.905565 1539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:42.905567 1541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:42.905659 1533 server_base.cc:1034] running on GCE node
W20260926 06:52:42.905613 1538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:42.905992 1533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:42.906229 1533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:42.907388 1533 hybrid_clock.cc:648] HybridClock initialized: now 1790405562907370 us; error 36 us; skew 500 ppm
Sep 26 06:52:42 dist-test-slave-ws74 krb5kdc[1292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405562, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:42.909816 1533 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:52:42.910166 1533 webserver.cc:460] Webserver started at http://127.0.191.194:32949/ using document root <none> and password file <none>
I20260926 06:52:42.910333 1533 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:42.910372 1533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:42.910454 1533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:42.911381 1533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "d1e2b5e0929e4d11bdd486ad6b32e938"
format_stamp: "Formatted at 2026-09-26 06:52:42 on dist-test-slave-ws74"
server_key: "60ae9fff149395740138ec6471190366"
server_key_iv: "5ce5b9d0596cb1842115c12e00e92b1f"
server_key_version: "encryptionkey@0"
I20260926 06:52:42.911661 1533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "d1e2b5e0929e4d11bdd486ad6b32e938"
format_stamp: "Formatted at 2026-09-26 06:52:42 on dist-test-slave-ws74"
server_key: "60ae9fff149395740138ec6471190366"
server_key_iv: "5ce5b9d0596cb1842115c12e00e92b1f"
server_key_version: "encryptionkey@0"
I20260926 06:52:42.912847 1533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:52:42.913479 1548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:42.913623 1533 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:42.913714 1533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "d1e2b5e0929e4d11bdd486ad6b32e938"
format_stamp: "Formatted at 2026-09-26 06:52:42 on dist-test-slave-ws74"
server_key: "60ae9fff149395740138ec6471190366"
server_key_iv: "5ce5b9d0596cb1842115c12e00e92b1f"
server_key_version: "encryptionkey@0"
I20260926 06:52:42.913765 1533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:42.927884 1533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:42.930107 1533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:42.930214 1533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:42.930493 1533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:52:42.930800 1533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:42.930835 1533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:42.930855 1533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:42.930902 1533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:42.936363 1533 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:34641
I20260926 06:52:42.936414 1661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:34641 every 8 connection(s)
I20260926 06:52:42.936712 1533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:52:42.941321 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 1533
I20260926 06:52:42.941437 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:52:42.941617 767 external_mini_cluster.cc:1428] Setting key 4a84b5d53eb9bf5e2b12c64e5b33294c
Sep 26 06:52:42 dist-test-slave-ws74 krb5kdc[1292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405562, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:42.945827 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:42.937386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:55437 (local address 127.0.191.254:33429)
0926 06:52:42.937451 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:42.937453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:42.937686 (+ 233us) server_negotiation.cc:407] Connection header received
0926 06:52:42.938121 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:42.938122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:42.938132 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:42.938212 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:42.939085 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:42.939580 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:42.940497 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:42.940683 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:42.940737 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:42.943030 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:42.943031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:42.943032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:42.943039 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:42.944442 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:42.944858 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:42.944859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:42.944860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:42.944921 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:42.945173 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:42.945174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:42.945175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:42.945285 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:42.945362 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:42.945652 (+ 290us) server_negotiation.cc:299] Negotiation successful
0926 06:52:42.945739 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 06:52:42.945999 1664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:42.937359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33429 (local address 127.0.191.194:55437)
0926 06:52:42.937563 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:42.937596 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:42.938039 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:42.938286 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:42.938289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:42.938829 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:42.939007 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:42.939011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:42.939652 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:42.939653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:42.940416 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:42.940418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:42.940757 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:42.940758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:42.940850 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:42.941325 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:42.941336 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:42.942936 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:42.944525 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:42.944530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:42.944531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:42.944776 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:42.944989 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:42.944990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:42.944990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:42.945103 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:42.945353 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:42.945359 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:42.945516 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 06:52:42.945625 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 06:52:42.945774 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":84,"threads_started":1}
I20260926 06:52:42.946559 1662 heartbeater.cc:347] Connected to a master server at 127.0.191.254:33429
I20260926 06:52:42.946660 1662 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:42.946863 1662 heartbeater.cc:511] Master 127.0.191.254:33429 requested a full tablet report, sending...
I20260926 06:52:42.947220 1337 ts_manager.cc:194] Registered new tserver with Master: d1e2b5e0929e4d11bdd486ad6b32e938 (127.0.191.194:34641)
I20260926 06:52:42.947598 1337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:55437
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:52:42.966537 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33429
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:43.046392 1667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:43.046591 1667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:43.046614 1667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:43.047768 1667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:43.047811 1667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:43.047837 1667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:52:43.049279 1667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33429
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:43.049523 1667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:43.049755 1667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:43.054648 1672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:43.054746 1675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:43.054657 1673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:43.054780 1667 server_base.cc:1034] running on GCE node
I20260926 06:52:43.054994 1667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:43.055220 1667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:43.056382 1667 hybrid_clock.cc:648] HybridClock initialized: now 1790405563056366 us; error 51 us; skew 500 ppm
Sep 26 06:52:43 dist-test-slave-ws74 krb5kdc[1292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405563, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:43.058794 1667 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:52:43.059203 1667 webserver.cc:460] Webserver started at http://127.0.191.195:45855/ using document root <none> and password file <none>
I20260926 06:52:43.059443 1667 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:43.059509 1667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:43.059659 1667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:43.060745 1667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "a9acfdbee82641da8b6e7acc5e944dd0"
format_stamp: "Formatted at 2026-09-26 06:52:43 on dist-test-slave-ws74"
server_key: "6d8caa9b4570f247e9f9026a61c748cf"
server_key_iv: "a166f9fbbdfadd4574588a97672722f7"
server_key_version: "encryptionkey@0"
I20260926 06:52:43.061084 1667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "a9acfdbee82641da8b6e7acc5e944dd0"
format_stamp: "Formatted at 2026-09-26 06:52:43 on dist-test-slave-ws74"
server_key: "6d8caa9b4570f247e9f9026a61c748cf"
server_key_iv: "a166f9fbbdfadd4574588a97672722f7"
server_key_version: "encryptionkey@0"
I20260926 06:52:43.062444 1667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:52:43.063185 1682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:43.063351 1667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:43.063446 1667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "a9acfdbee82641da8b6e7acc5e944dd0"
format_stamp: "Formatted at 2026-09-26 06:52:43 on dist-test-slave-ws74"
server_key: "6d8caa9b4570f247e9f9026a61c748cf"
server_key_iv: "a166f9fbbdfadd4574588a97672722f7"
server_key_version: "encryptionkey@0"
I20260926 06:52:43.063525 1667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:43.079746 1667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:43.081743 1667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:43.081895 1667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:43.082110 1667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:52:43.082402 1667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:43.082454 1667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:43.082494 1667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:43.082521 1667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:43.088137 1667 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:45753
I20260926 06:52:43.088207 1795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:45753 every 8 connection(s)
I20260926 06:52:43.088516 1667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:52:43.090960 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 1667
I20260926 06:52:43.091054 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:52:43.091212 767 external_mini_cluster.cc:1428] Setting key 47a680b16f5ad86dc3d328404bed62e5
Sep 26 06:52:43 dist-test-slave-ws74 krb5kdc[1292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405563, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:43.097755 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.089221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:55317 (local address 127.0.191.254:33429)
0926 06:52:43.089267 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:43.089270 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:43.089548 (+ 278us) server_negotiation.cc:407] Connection header received
0926 06:52:43.089997 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:43.089998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:43.090008 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:43.090060 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:43.090972 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.091659 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.092771 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.092979 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.093069 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.095125 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:43.095126 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:43.095127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:43.095134 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:43.096311 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:43.096739 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:43.096741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:43.096741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:43.096802 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:43.097060 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:43.097061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:43.097062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:43.097178 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:43.097255 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:43.097579 (+ 324us) server_negotiation.cc:299] Negotiation successful
0926 06:52:43.097663 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:52:43.097936 1798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.089227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33429 (local address 127.0.191.195:55317)
0926 06:52:43.089367 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:43.089448 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:43.089919 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:43.090112 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:43.090116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:43.090645 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:43.090887 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:43.090893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.091752 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.091753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:43.092666 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:43.092668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.093077 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.093078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:43.093210 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.093745 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:43.093753 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:43.095005 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:43.096423 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:43.096429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:43.096432 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:43.096647 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:43.096868 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:43.096869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:43.096870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:43.096975 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:43.097245 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:43.097250 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:43.097429 (+ 179us) client_negotiation.cc:769] Sending connection context
0926 06:52:43.097560 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 06:52:43.097710 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
I20260926 06:52:43.098368 1796 heartbeater.cc:347] Connected to a master server at 127.0.191.254:33429
I20260926 06:52:43.098441 1796 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:43.098634 1796 heartbeater.cc:511] Master 127.0.191.254:33429 requested a full tablet report, sending...
I20260926 06:52:43.098966 1337 ts_manager.cc:194] Registered new tserver with Master: a9acfdbee82641da8b6e7acc5e944dd0 (127.0.191.195:45753)
I20260926 06:52:43.099294 1337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:55317
I20260926 06:52:43.103051 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:52:43.110118 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.104726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49836 (local address 127.0.191.254:33429)
0926 06:52:43.104777 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:43.104778 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:43.104870 (+ 92us) server_negotiation.cc:407] Connection header received
0926 06:52:43.104920 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:43.104920 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:43.104929 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:43.104979 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:43.105535 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.106000 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.106746 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.106915 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.106963 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.108016 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:43.108017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:43.108017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:43.108023 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:43.109161 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:43.109515 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:43.109515 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:43.109516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:43.109560 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:43.109699 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:43.109699 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:43.109699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:43.109823 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:43.109891 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:43.109981 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 06:52:43.110036 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 06:52:43.112030 1337 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:52:43.113523 1337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:52:43.120271 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.115957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47202 (local address 127.0.191.193:45037)
0926 06:52:43.116195 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:43.116201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:43.116211 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:52:43.116273 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:43.116276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:43.116293 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:43.116365 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:43.116506 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.117687 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.119070 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.119777 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.119849 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.119908 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:43.120145 (+ 237us) server_negotiation.cc:299] Negotiation successful
0926 06:52:43.120186 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
I20260926 06:52:43.120265 1804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.115679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:45753 (local address 127.0.0.1:55910)
0926 06:52:43.115797 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:43.115832 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:43.115897 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:43.116321 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:43.116322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:43.116324 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:43.116409 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:43.116411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.117782 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.117782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:43.119143 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:43.119145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.119957 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.119958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:43.120054 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.120056 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:52:43.120093 (+ 37us) client_negotiation.cc:240] Negotiation successful
0926 06:52:43.120144 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":48,"thread_start_us":76,"threads_started":1}
I20260926 06:52:43.120430 1808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.115954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54454 (local address 127.0.191.194:34641)
0926 06:52:43.116162 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:43.116166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:43.116175 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:52:43.116210 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:43.116212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:43.116225 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:43.116301 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:43.116491 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.117578 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.119391 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.119865 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.119950 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.120004 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:43.120303 (+ 299us) server_negotiation.cc:299] Negotiation successful
0926 06:52:43.120355 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":85,"threads_started":1}
I20260926 06:52:43.120343 1805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.115691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:34641 (local address 127.0.0.1:54454)
0926 06:52:43.115990 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:43.115999 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:43.116037 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:43.116317 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:43.116320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:43.116322 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:43.116412 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:43.116414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.117683 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.117684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:43.119311 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:43.119313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.119948 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.119949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:43.120059 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.120063 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:52:43.120140 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 06:52:43.120181 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":137,"thread_start_us":65,"threads_started":1}
I20260926 06:52:43.120278 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.115694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:45037 (local address 127.0.0.1:47202)
0926 06:52:43.116100 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:43.116110 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:43.116145 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:43.116389 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:43.116390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:43.116392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:43.116422 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:43.116423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.117801 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.117802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:43.118983 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:43.118985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.119944 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.119945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:43.120031 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.120044 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 06:52:43.120119 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:52:43.120147 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":58,"thread_start_us":56,"threads_started":1}
I20260926 06:52:43.121060 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.115940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55910 (local address 127.0.191.195:45753)
0926 06:52:43.116129 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:43.116133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:43.116145 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:52:43.116195 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:43.116198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:43.116214 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:43.116489 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:43.116563 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.117691 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.119221 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.119872 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.120732 (+ 860us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.120793 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:43.120816 (+ 23us) server_negotiation.cc:299] Negotiation successful
0926 06:52:43.120860 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":80,"threads_started":1}
I20260926 06:52:43.121295 1596 tablet_service.cc:1467] Processing CreateTablet for tablet c6e3312fbd22427baaf8f58a025eef34 (DEFAULT_TABLE table=test-table [id=053c2c06e4e243c29ee81aa66492ced2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:43.121551 1462 tablet_service.cc:1467] Processing CreateTablet for tablet c6e3312fbd22427baaf8f58a025eef34 (DEFAULT_TABLE table=test-table [id=053c2c06e4e243c29ee81aa66492ced2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:43.121675 1596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e3312fbd22427baaf8f58a025eef34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:43.121887 1462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e3312fbd22427baaf8f58a025eef34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:43.121958 1730 tablet_service.cc:1467] Processing CreateTablet for tablet c6e3312fbd22427baaf8f58a025eef34 (DEFAULT_TABLE table=test-table [id=053c2c06e4e243c29ee81aa66492ced2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:43.122251 1730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e3312fbd22427baaf8f58a025eef34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:43.124483 1810 tablet_bootstrap.cc:492] T c6e3312fbd22427baaf8f58a025eef34 P 17933da43d3542e695852213175df76f: Bootstrap starting.
I20260926 06:52:43.124946 1811 tablet_bootstrap.cc:492] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938: Bootstrap starting.
I20260926 06:52:43.125278 1810 tablet_bootstrap.cc:654] T c6e3312fbd22427baaf8f58a025eef34 P 17933da43d3542e695852213175df76f: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:43.125334 1812 tablet_bootstrap.cc:492] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0: Bootstrap starting.
I20260926 06:52:43.125598 1811 tablet_bootstrap.cc:654] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:43.125945 1812 tablet_bootstrap.cc:654] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:43.126165 1810 log.cc:824] T c6e3312fbd22427baaf8f58a025eef34 P 17933da43d3542e695852213175df76f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:43.126165 1811 log.cc:824] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:43.126518 1812 log.cc:824] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:43.126938 1811 tablet_bootstrap.cc:492] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938: No bootstrap required, opened a new log
I20260926 06:52:43.127017 1811 ts_tablet_manager.cc:1397] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:52:43.127280 1812 tablet_bootstrap.cc:492] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0: No bootstrap required, opened a new log
I20260926 06:52:43.127329 1812 ts_tablet_manager.cc:1397] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:52:43.128388 1811 raft_consensus.cc:348] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17933da43d3542e695852213175df76f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45037 } } peers { permanent_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45753 } } peers { permanent_uuid: "d1e2b5e0929e4d11bdd486ad6b32e938" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 34641 } }
I20260926 06:52:43.128553 1811 raft_consensus.cc:374] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:43.128599 1811 raft_consensus.cc:729] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1e2b5e0929e4d11bdd486ad6b32e938, State: Initialized, Role: FOLLOWER
I20260926 06:52:43.128607 1812 raft_consensus.cc:348] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17933da43d3542e695852213175df76f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45037 } } peers { permanent_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45753 } } peers { permanent_uuid: "d1e2b5e0929e4d11bdd486ad6b32e938" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 34641 } }
I20260926 06:52:43.128711 1811 consensus_queue.cc:260] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17933da43d3542e695852213175df76f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45037 } } peers { permanent_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45753 } } peers { permanent_uuid: "d1e2b5e0929e4d11bdd486ad6b32e938" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 34641 } }
I20260926 06:52:43.128768 1812 raft_consensus.cc:374] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:43.128823 1812 raft_consensus.cc:729] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9acfdbee82641da8b6e7acc5e944dd0, State: Initialized, Role: FOLLOWER
I20260926 06:52:43.128930 1812 consensus_queue.cc:260] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17933da43d3542e695852213175df76f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45037 } } peers { permanent_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45753 } } peers { permanent_uuid: "d1e2b5e0929e4d11bdd486ad6b32e938" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 34641 } }
I20260926 06:52:43.128984 1811 ts_tablet_manager.cc:1428] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:52:43.129089 1662 heartbeater.cc:503] Master 127.0.191.254:33429 was elected leader, sending a full tablet report...
I20260926 06:52:43.129196 1812 ts_tablet_manager.cc:1428] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:52:43.129273 1796 heartbeater.cc:503] Master 127.0.191.254:33429 was elected leader, sending a full tablet report...
I20260926 06:52:43.129695 1810 tablet_bootstrap.cc:492] T c6e3312fbd22427baaf8f58a025eef34 P 17933da43d3542e695852213175df76f: No bootstrap required, opened a new log
I20260926 06:52:43.129781 1810 ts_tablet_manager.cc:1397] T c6e3312fbd22427baaf8f58a025eef34 P 17933da43d3542e695852213175df76f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260926 06:52:43.131368 1810 raft_consensus.cc:348] T c6e3312fbd22427baaf8f58a025eef34 P 17933da43d3542e695852213175df76f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17933da43d3542e695852213175df76f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45037 } } peers { permanent_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45753 } } peers { permanent_uuid: "d1e2b5e0929e4d11bdd486ad6b32e938" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 34641 } }
I20260926 06:52:43.131490 1810 raft_consensus.cc:374] T c6e3312fbd22427baaf8f58a025eef34 P 17933da43d3542e695852213175df76f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:43.131511 1810 raft_consensus.cc:729] T c6e3312fbd22427baaf8f58a025eef34 P 17933da43d3542e695852213175df76f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17933da43d3542e695852213175df76f, State: Initialized, Role: FOLLOWER
I20260926 06:52:43.131613 1810 consensus_queue.cc:260] T c6e3312fbd22427baaf8f58a025eef34 P 17933da43d3542e695852213175df76f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17933da43d3542e695852213175df76f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45037 } } peers { permanent_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45753 } } peers { permanent_uuid: "d1e2b5e0929e4d11bdd486ad6b32e938" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 34641 } }
I20260926 06:52:43.131887 1810 ts_tablet_manager.cc:1428] T c6e3312fbd22427baaf8f58a025eef34 P 17933da43d3542e695852213175df76f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:52:43.131891 1528 heartbeater.cc:503] Master 127.0.191.254:33429 was elected leader, sending a full tablet report...
W20260926 06:52:43.187247 1663 tablet.cc:2369] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:52:43.221741 1817 raft_consensus.cc:482] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:52:43.221868 1817 raft_consensus.cc:504] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17933da43d3542e695852213175df76f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45037 } } peers { permanent_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45753 } } peers { permanent_uuid: "d1e2b5e0929e4d11bdd486ad6b32e938" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 34641 } }
I20260926 06:52:43.222306 1817 leader_election.cc:288] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17933da43d3542e695852213175df76f (127.0.191.193:45037), d1e2b5e0929e4d11bdd486ad6b32e938 (127.0.191.194:34641)
I20260926 06:52:43.226166 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.222562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:55799 (local address 127.0.191.193:45037)
0926 06:52:43.222603 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:43.222605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:43.222714 (+ 109us) server_negotiation.cc:407] Connection header received
0926 06:52:43.222748 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:43.222749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:43.222761 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:43.222813 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:43.223002 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.223751 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.225222 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.225820 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.225893 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.225901 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:43.226061 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 06:52:43.226097 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:52:43.226166 1808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.222555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:55195 (local address 127.0.191.194:34641)
0926 06:52:43.222601 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:43.222604 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:43.222616 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:52:43.222676 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:43.222677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:43.222704 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:43.222781 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:43.223004 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.223741 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.225222 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.225820 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.225894 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.225901 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:43.226061 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 06:52:43.226096 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:52:43.226181 1798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.222485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:34641 (local address 127.0.191.195:55195)
0926 06:52:43.222552 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:43.222565 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:43.222608 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:43.222819 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:43.222819 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:43.222821 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:43.222919 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:43.222921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.223821 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.223822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:43.225135 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:43.225138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.225903 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.225904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:43.225986 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.225987 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:52:43.226043 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 06:52:43.226087 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":77}
I20260926 06:52:43.226189 1819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.222485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:45037 (local address 127.0.191.195:55799)
0926 06:52:43.222648 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:43.222656 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:43.222711 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:43.222823 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:43.222823 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:43.222824 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:43.222919 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:43.222921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.223821 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.223822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:43.225135 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:43.225138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.225905 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.225906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:43.225982 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.225987 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:52:43.226043 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 06:52:43.226087 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":111,"thread_start_us":85,"threads_started":1}
I20260926 06:52:43.226558 1616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e3312fbd22427baaf8f58a025eef34" candidate_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1e2b5e0929e4d11bdd486ad6b32e938" is_pre_election: true
I20260926 06:52:43.226589 1482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e3312fbd22427baaf8f58a025eef34" candidate_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17933da43d3542e695852213175df76f" is_pre_election: true
I20260926 06:52:43.226716 1482 raft_consensus.cc:2454] T c6e3312fbd22427baaf8f58a025eef34 P 17933da43d3542e695852213175df76f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9acfdbee82641da8b6e7acc5e944dd0 in term 0.
I20260926 06:52:43.226720 1616 raft_consensus.cc:2454] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9acfdbee82641da8b6e7acc5e944dd0 in term 0.
I20260926 06:52:43.226935 1684 leader_election.cc:302] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17933da43d3542e695852213175df76f, a9acfdbee82641da8b6e7acc5e944dd0; no voters:
I20260926 06:52:43.227144 1817 raft_consensus.cc:2781] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:52:43.227198 1817 raft_consensus.cc:482] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:52:43.227222 1817 raft_consensus.cc:3037] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:43.227883 1817 raft_consensus.cc:504] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17933da43d3542e695852213175df76f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45037 } } peers { permanent_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45753 } } peers { permanent_uuid: "d1e2b5e0929e4d11bdd486ad6b32e938" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 34641 } }
I20260926 06:52:43.228015 1817 leader_election.cc:288] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [CANDIDATE]: Term 1 election: Requested vote from peers 17933da43d3542e695852213175df76f (127.0.191.193:45037), d1e2b5e0929e4d11bdd486ad6b32e938 (127.0.191.194:34641)
I20260926 06:52:43.228196 1482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e3312fbd22427baaf8f58a025eef34" candidate_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17933da43d3542e695852213175df76f"
I20260926 06:52:43.228211 1616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e3312fbd22427baaf8f58a025eef34" candidate_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1e2b5e0929e4d11bdd486ad6b32e938"
I20260926 06:52:43.228268 1482 raft_consensus.cc:3037] T c6e3312fbd22427baaf8f58a025eef34 P 17933da43d3542e695852213175df76f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:43.228332 1616 raft_consensus.cc:3037] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:43.229038 1482 raft_consensus.cc:2454] T c6e3312fbd22427baaf8f58a025eef34 P 17933da43d3542e695852213175df76f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9acfdbee82641da8b6e7acc5e944dd0 in term 1.
I20260926 06:52:43.229053 1616 raft_consensus.cc:2454] T c6e3312fbd22427baaf8f58a025eef34 P d1e2b5e0929e4d11bdd486ad6b32e938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9acfdbee82641da8b6e7acc5e944dd0 in term 1.
I20260926 06:52:43.229200 1684 leader_election.cc:302] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17933da43d3542e695852213175df76f, a9acfdbee82641da8b6e7acc5e944dd0; no voters:
I20260926 06:52:43.229326 1817 raft_consensus.cc:2781] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:43.229506 1817 raft_consensus.cc:686] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [term 1 LEADER]: Becoming Leader. State: Replica: a9acfdbee82641da8b6e7acc5e944dd0, State: Running, Role: LEADER
I20260926 06:52:43.229620 1817 consensus_queue.cc:237] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17933da43d3542e695852213175df76f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45037 } } peers { permanent_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45753 } } peers { permanent_uuid: "d1e2b5e0929e4d11bdd486ad6b32e938" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 34641 } }
I20260926 06:52:43.230443 1336 catalog_manager.cc:5495] T c6e3312fbd22427baaf8f58a025eef34 P a9acfdbee82641da8b6e7acc5e944dd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9acfdbee82641da8b6e7acc5e944dd0 (127.0.191.195). New cstate: current_term: 1 leader_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17933da43d3542e695852213175df76f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9acfdbee82641da8b6e7acc5e944dd0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1e2b5e0929e4d11bdd486ad6b32e938" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 34641 } health_report { overall_health: UNKNOWN } } }
Sep 26 06:52:43 dist-test-slave-ws74 krb5kdc[1292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.193@KRBTEST.COM
I20260926 06:52:43.239464 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.232787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47208 (local address 127.0.191.193:45037)
0926 06:52:43.232828 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:43.232830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:43.232975 (+ 145us) server_negotiation.cc:407] Connection header received
0926 06:52:43.233011 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:43.233012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:43.233021 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:43.233071 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:43.233610 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.234051 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.234690 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.234863 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.234913 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.237019 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:43.237026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:43.237026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:43.237032 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:43.238456 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:43.238779 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:43.238783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:43.238783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:43.238843 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:43.239010 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:43.239011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:43.239011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:43.239135 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:43.239234 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:43.239297 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 06:52:43.239356 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 26 06:52:43 dist-test-slave-ws74 krb5kdc[1292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405563, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 06:52:43 dist-test-slave-ws74 krb5kdc[1292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405563, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.191.193@KRBTEST.COM
I20260926 06:52:43.261422 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.254936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.0.191.193:45037)
0926 06:52:43.255011 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:43.255013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:43.255090 (+ 77us) server_negotiation.cc:407] Connection header received
0926 06:52:43.255125 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:43.255126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:43.255134 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:43.255181 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:43.255654 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.256123 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.256788 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.256990 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.257045 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.259137 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:43.259138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:43.259138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:43.259144 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:43.260304 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:43.260596 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:43.260597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:43.260597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:43.260644 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:43.260853 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:43.260854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:43.260855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:43.260952 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:43.261051 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:43.261218 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 06:52:43.261290 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Sep 26 06:52:43 dist-test-slave-ws74 krb5kdc[1292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 06:52:43 dist-test-slave-ws74 krb5kdc[1292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.193@KRBTEST.COM
I20260926 06:52:43.282172 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:43.275501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47224 (local address 127.0.191.193:45037)
0926 06:52:43.275573 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:43.275575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:43.275669 (+ 94us) server_negotiation.cc:407] Connection header received
0926 06:52:43.275753 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:43.275754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:43.275764 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:43.275831 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:43.276366 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.276821 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.277595 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:43.277772 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:43.277857 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:43.279876 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:43.279877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:43.279878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:43.279885 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:43.281015 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:43.281394 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:43.281395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:43.281395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:43.281453 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:43.281646 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:43.281647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:43.281648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:43.281746 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:43.281845 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:43.281989 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 06:52:43.282068 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260926 06:52:43.282624 1442 tablet_service.cc:2908] Rejecting scan request for tablet c6e3312fbd22427baaf8f58a025eef34: Uninitialized: clean time has not yet been initialized
I20260926 06:52:43.283378 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 1399
I20260926 06:52:43.288501 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 1533
I20260926 06:52:43.293239 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 1667
I20260926 06:52:43.297504 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 1305
2026-09-26T06:52:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1987 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:52:43 dist-test-slave-ws74 krb5kdc[1834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:52:43 dist-test-slave-ws74 krb5kdc[1834](info): set up 2 sockets
Sep 26 06:52:43 dist-test-slave-ws74 krb5kdc[1834](info): commencing operation
krb5kdc: starting...
W20260926 06:52:44.324935 767 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:52:44 dist-test-slave-ws74 krb5kdc[1834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:52:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:52:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:52:44.435853 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:44667
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:37255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:44667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:44.513168 1847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:44.513358 1847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:44.513432 1847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:44.514590 1847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:52:44.514637 1847 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 06:52:44.514652 1847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:44.514704 1847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:44.514722 1847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:52:44.514747 1847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:52:44.516215 1847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:37255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:44667
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:44667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:44.516454 1847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:44.516680 1847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:44.521502 1855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:44.521481 1853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:44.521522 1847 server_base.cc:1034] running on GCE node
W20260926 06:52:44.521495 1852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:44.521939 1847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:44.522246 1847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:44.523389 1847 hybrid_clock.cc:648] HybridClock initialized: now 1790405564523372 us; error 40 us; skew 500 ppm
Sep 26 06:52:44 dist-test-slave-ws74 krb5kdc[1834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405564, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:44.525677 1847 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:52:44.526087 1847 webserver.cc:460] Webserver started at http://127.0.191.254:41963/ using document root <none> and password file <none>
I20260926 06:52:44.526321 1847 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:44.526364 1847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:44.526511 1847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:44.527540 1847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "f86f3ab5ccbf4e7c99099d372cb28874"
format_stamp: "Formatted at 2026-09-26 06:52:44 on dist-test-slave-ws74"
server_key: "4065bd8ea11a6cbf07f182c7e3af9d43"
server_key_iv: "73bb42d1e444f30e3e0366dcf0128347"
server_key_version: "encryptionkey@0"
I20260926 06:52:44.527884 1847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "f86f3ab5ccbf4e7c99099d372cb28874"
format_stamp: "Formatted at 2026-09-26 06:52:44 on dist-test-slave-ws74"
server_key: "4065bd8ea11a6cbf07f182c7e3af9d43"
server_key_iv: "73bb42d1e444f30e3e0366dcf0128347"
server_key_version: "encryptionkey@0"
I20260926 06:52:44.529124 1847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:52:44.529887 1862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:44.530063 1847 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:44.530153 1847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "f86f3ab5ccbf4e7c99099d372cb28874"
format_stamp: "Formatted at 2026-09-26 06:52:44 on dist-test-slave-ws74"
server_key: "4065bd8ea11a6cbf07f182c7e3af9d43"
server_key_iv: "73bb42d1e444f30e3e0366dcf0128347"
server_key_version: "encryptionkey@0"
I20260926 06:52:44.530221 1847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:44.548200 1847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:44.550199 1847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:44.550341 1847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:44.554631 1847 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:44667
I20260926 06:52:44.554728 1924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:44667 every 8 connection(s)
I20260926 06:52:44.555017 1847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:52:44.555707 1925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:44.557946 1925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874: Bootstrap starting.
I20260926 06:52:44.558548 1925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:44.558832 1925 log.cc:824] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:44.559444 1925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874: No bootstrap required, opened a new log
I20260926 06:52:44.559953 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 1847
I20260926 06:52:44.560051 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:52:44.560211 767 external_mini_cluster.cc:1428] Setting key 6a4f97a48b3046952ddba8edc985b769
I20260926 06:52:44.561981 1925 raft_consensus.cc:348] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f86f3ab5ccbf4e7c99099d372cb28874" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44667 } }
I20260926 06:52:44.562145 1925 raft_consensus.cc:374] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:44.562186 1925 raft_consensus.cc:729] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f86f3ab5ccbf4e7c99099d372cb28874, State: Initialized, Role: FOLLOWER
I20260926 06:52:44.562340 1925 consensus_queue.cc:260] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f86f3ab5ccbf4e7c99099d372cb28874" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44667 } }
I20260926 06:52:44.562433 1925 raft_consensus.cc:388] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:52:44.562469 1925 raft_consensus.cc:482] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:52:44.562534 1925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:44.563156 1925 raft_consensus.cc:504] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f86f3ab5ccbf4e7c99099d372cb28874" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44667 } }
I20260926 06:52:44.563241 1925 leader_election.cc:302] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f86f3ab5ccbf4e7c99099d372cb28874; no voters:
I20260926 06:52:44.563421 1925 leader_election.cc:288] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:52:44.563459 1930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:44.563582 1930 raft_consensus.cc:686] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [term 1 LEADER]: Becoming Leader. State: Replica: f86f3ab5ccbf4e7c99099d372cb28874, State: Running, Role: LEADER
I20260926 06:52:44.563671 1925 sys_catalog.cc:564] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:52:44.563659 1930 consensus_queue.cc:237] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f86f3ab5ccbf4e7c99099d372cb28874" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44667 } }
I20260926 06:52:44.563934 1930 sys_catalog.cc:455] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f86f3ab5ccbf4e7c99099d372cb28874. Latest consensus state: current_term: 1 leader_uuid: "f86f3ab5ccbf4e7c99099d372cb28874" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f86f3ab5ccbf4e7c99099d372cb28874" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44667 } } }
I20260926 06:52:44.564011 1930 sys_catalog.cc:458] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:44.563966 1932 sys_catalog.cc:455] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f86f3ab5ccbf4e7c99099d372cb28874" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f86f3ab5ccbf4e7c99099d372cb28874" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44667 } } }
I20260926 06:52:44.564182 1932 sys_catalog.cc:458] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874 [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:44.565004 1935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:52:44.565743 1935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:52:44.567297 1935 catalog_manager.cc:1347] Generated new cluster ID: 62464ee2bd284f69a7b6a745b250e98f
I20260926 06:52:44.567346 1935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:52:44 dist-test-slave-ws74 krb5kdc[1834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:44.571316 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:44.560668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59448 (local address 127.0.191.254:44667)
0926 06:52:44.560869 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:44.560876 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:44.560899 (+ 23us) server_negotiation.cc:407] Connection header received
0926 06:52:44.561963 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:44.561969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:44.562014 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:44.562124 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:44.562884 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.564092 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.565044 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.565659 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.565793 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:44.568263 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:44.568267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:44.568267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:44.568278 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:44.569815 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:44.570179 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:44.570182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:44.570182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:44.570283 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:44.570483 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:44.570484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:44.570485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:44.570723 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:44.570885 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:44.570977 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 06:52:44.571111 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":118,"threads_started":1}
I20260926 06:52:44.589254 1935 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:52:44.589993 1935 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:52:44.595297 1935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874: Generated new TSK 0
I20260926 06:52:44.595474 1935 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:52:44.626422 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:44667
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:37255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:44.704232 1951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:44.704416 1951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:44.704437 1951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:44.705608 1951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:44.705647 1951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:44.705667 1951 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:52:44.705716 1951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:52:44.707161 1951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:37255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.191.254:44667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:44.707399 1951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:44.707612 1951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:44.712322 1956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:44.712373 1951 server_base.cc:1034] running on GCE node
W20260926 06:52:44.712343 1959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:44.712355 1957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:44.712767 1951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:44.712961 1951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:44.714097 1951 hybrid_clock.cc:648] HybridClock initialized: now 1790405564714082 us; error 30 us; skew 500 ppm
Sep 26 06:52:44 dist-test-slave-ws74 krb5kdc[1834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405564, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:44.716449 1951 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:52:44.716866 1951 webserver.cc:460] Webserver started at http://127.0.191.193:46783/ using document root <none> and password file <none>
I20260926 06:52:44.717103 1951 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:44.717151 1951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:44.717320 1951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:44.718426 1951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "20f6eafd243c466a946da5e79675845e"
format_stamp: "Formatted at 2026-09-26 06:52:44 on dist-test-slave-ws74"
server_key: "47fd236841b58b28286818e68a0773fe"
server_key_iv: "090b82902703eae72ed4cbbd303adefa"
server_key_version: "encryptionkey@0"
I20260926 06:52:44.718786 1951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "20f6eafd243c466a946da5e79675845e"
format_stamp: "Formatted at 2026-09-26 06:52:44 on dist-test-slave-ws74"
server_key: "47fd236841b58b28286818e68a0773fe"
server_key_iv: "090b82902703eae72ed4cbbd303adefa"
server_key_version: "encryptionkey@0"
I20260926 06:52:44.720049 1951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:52:44.720744 1966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:44.720937 1951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:52:44.721024 1951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "20f6eafd243c466a946da5e79675845e"
format_stamp: "Formatted at 2026-09-26 06:52:44 on dist-test-slave-ws74"
server_key: "47fd236841b58b28286818e68a0773fe"
server_key_iv: "090b82902703eae72ed4cbbd303adefa"
server_key_version: "encryptionkey@0"
I20260926 06:52:44.721096 1951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:44.734009 1951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:44.735682 1951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:44.735828 1951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:44.736299 1951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:44.736337 1951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:44.736359 1951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:44.736411 1951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 06:52:44 dist-test-slave-ws74 krb5kdc[1834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405564, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:44.743439 1951 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:40009
I20260926 06:52:44.743547 2081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:40009 every 8 connection(s)
I20260926 06:52:44.743973 1951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:52:44.746433 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:44.736781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:44421 (local address 127.0.191.254:44667)
0926 06:52:44.736893 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:44.736896 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:44.737033 (+ 137us) server_negotiation.cc:407] Connection header received
0926 06:52:44.737516 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:44.737517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:44.737535 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:44.737612 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:44.738402 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.739253 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.740778 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.741127 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.741195 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:44.743531 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:44.743532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:44.743533 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:44.743540 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:44.745067 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:44.745473 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:44.745474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:44.745475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:44.745529 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:44.745785 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:44.745786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:44.745786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:44.745894 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:44.745984 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:44.746259 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 06:52:44.746344 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 06:52:44.746570 1975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:44.736713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:44667 (local address 127.0.191.193:44421)
0926 06:52:44.736905 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:44.736941 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:44.737421 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:44.737670 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:44.737673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:44.738016 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:44.738276 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:44.738281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.739361 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.739362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:44.740690 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:44.740693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.741271 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.741273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:44.741437 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:44.741942 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:44.741969 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:44.743450 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:44.745164 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:44.745170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:44.745173 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:44.745416 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:44.745610 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:44.745611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:44.745611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:44.745717 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:44.745975 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:44.745981 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:44.746110 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 06:52:44.746219 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 06:52:44.746364 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
I20260926 06:52:44.747043 2082 heartbeater.cc:347] Connected to a master server at 127.0.191.254:44667
I20260926 06:52:44.747133 2082 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:44.747305 2082 heartbeater.cc:511] Master 127.0.191.254:44667 requested a full tablet report, sending...
I20260926 06:52:44.747843 1878 ts_manager.cc:194] Registered new tserver with Master: 20f6eafd243c466a946da5e79675845e (127.0.191.193:40009)
I20260926 06:52:44.748766 1878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:44421
I20260926 06:52:44.750808 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 1951
I20260926 06:52:44.750880 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:52:44.751067 767 external_mini_cluster.cc:1428] Setting key 6dd709426b9fa102024232cca02d59d4
I20260926 06:52:44.753669 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:44.748045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:50555 (local address 127.0.191.254:44667)
0926 06:52:44.748082 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:44.748084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:44.748214 (+ 130us) server_negotiation.cc:407] Connection header received
0926 06:52:44.748265 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:44.748266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:44.748274 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:44.748313 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:44.748641 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.749266 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.750036 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.750221 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.750273 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:44.751034 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:44.751035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:44.751036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:44.751042 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:44.752550 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:44.752861 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:44.752862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:44.752862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:44.752919 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:44.753111 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:44.753113 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:44.753113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:44.753221 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:44.753295 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:44.753507 (+ 212us) server_negotiation.cc:299] Negotiation successful
0926 06:52:44.753575 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:52:44.753676 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:44.747999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:44667 (local address 127.0.191.193:50555)
0926 06:52:44.748141 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:44.748151 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:44.748211 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:44.748324 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:44.748325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:44.748503 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:44.748554 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:44.748555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.749360 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.749361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:44.749968 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:44.749970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.750284 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.750284 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:44.750328 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:44.750648 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:44.750652 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:44.750946 (+ 294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:44.752629 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:44.752630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:44.752630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:44.752791 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:44.752982 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:44.752982 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:44.752983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:44.753031 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:44.753299 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:44.753300 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:44.753346 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 06:52:44.753479 (+ 133us) client_negotiation.cc:240] Negotiation successful
0926 06:52:44.753575 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:52:44.775238 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:44667
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:37255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:44.855139 2091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:44.855322 2091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:44.855345 2091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:44.856387 2091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:44.856418 2091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:44.856438 2091 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:52:44.856455 2091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:52:44.857893 2091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:37255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.191.254:44667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:44.858069 2091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:44.858237 2091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:44.862977 2099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:44.863031 2097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:44.862978 2096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:44.863225 2091 server_base.cc:1034] running on GCE node
I20260926 06:52:44.863436 2091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:44.863678 2091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:44.864835 2091 hybrid_clock.cc:648] HybridClock initialized: now 1790405564864820 us; error 28 us; skew 500 ppm
Sep 26 06:52:44 dist-test-slave-ws74 krb5kdc[1834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405564, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:44.867151 2091 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:52:44.867638 2091 webserver.cc:460] Webserver started at http://127.0.191.194:41997/ using document root <none> and password file <none>
I20260926 06:52:44.867868 2091 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:44.867936 2091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:44.868080 2091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:44.869148 2091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "73bc817f2231454c8eb7a1545535452d"
format_stamp: "Formatted at 2026-09-26 06:52:44 on dist-test-slave-ws74"
server_key: "a6db4429b67db1c8a2a22372a68d289f"
server_key_iv: "f6fa58335256026715ab294ab7f38111"
server_key_version: "encryptionkey@0"
I20260926 06:52:44.869534 2091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "73bc817f2231454c8eb7a1545535452d"
format_stamp: "Formatted at 2026-09-26 06:52:44 on dist-test-slave-ws74"
server_key: "a6db4429b67db1c8a2a22372a68d289f"
server_key_iv: "f6fa58335256026715ab294ab7f38111"
server_key_version: "encryptionkey@0"
I20260926 06:52:44.870781 2091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:52:44.871451 2106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:44.871726 2091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:52:44.871821 2091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "73bc817f2231454c8eb7a1545535452d"
format_stamp: "Formatted at 2026-09-26 06:52:44 on dist-test-slave-ws74"
server_key: "a6db4429b67db1c8a2a22372a68d289f"
server_key_iv: "f6fa58335256026715ab294ab7f38111"
server_key_version: "encryptionkey@0"
I20260926 06:52:44.871901 2091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:44.879099 2091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:44.880936 2091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:44.881094 2091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:44.881691 2091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:44.881752 2091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:44.881793 2091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:44.881824 2091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 06:52:44 dist-test-slave-ws74 krb5kdc[1834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405564, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:44.888950 2091 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:40059
I20260926 06:52:44.889328 2091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:52:44.889530 2221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:40059 every 8 connection(s)
I20260926 06:52:44.890630 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:44.882041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:58989 (local address 127.0.191.254:44667)
0926 06:52:44.882117 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:44.882119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:44.882300 (+ 181us) server_negotiation.cc:407] Connection header received
0926 06:52:44.882754 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:44.882755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:44.882769 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:44.882847 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:44.883407 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.883907 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.884952 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.885168 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.885251 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:44.887955 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:44.887957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:44.887957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:44.887966 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:44.889509 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:44.889980 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:44.889981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:44.889981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:44.890033 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:44.890204 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:44.890205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:44.890205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:44.890296 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:44.890340 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:44.890514 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 06:52:44.890569 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 06:52:44.890797 2115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:44.882035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:44667 (local address 127.0.191.194:58989)
0926 06:52:44.882169 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:44.882207 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:44.882653 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:44.882897 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:44.882900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:44.883130 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:44.883295 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:44.883323 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.884019 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.884020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:44.884853 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:44.884856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.885268 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.885270 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:44.885406 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:44.885897 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:44.885910 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:44.887849 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:44.889614 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:44.889618 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:44.889620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:44.889921 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:44.890088 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:44.890089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:44.890089 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:44.890170 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:44.890331 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:44.890333 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:44.890430 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 06:52:44.890501 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 06:52:44.890607 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":69,"threads_started":1}
I20260926 06:52:44.891301 2222 heartbeater.cc:347] Connected to a master server at 127.0.191.254:44667
I20260926 06:52:44.891391 2222 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:44.891552 2222 heartbeater.cc:511] Master 127.0.191.254:44667 requested a full tablet report, sending...
I20260926 06:52:44.891934 1879 ts_manager.cc:194] Registered new tserver with Master: 73bc817f2231454c8eb7a1545535452d (127.0.191.194:40059)
I20260926 06:52:44.892449 1879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:58989
I20260926 06:52:44.897797 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:44.892269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:44667 (local address 127.0.191.194:43051)
0926 06:52:44.892376 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:44.892386 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:44.892443 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:44.892559 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:44.892560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:44.892772 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:44.892839 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:44.892840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.893497 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.893498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:44.894015 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:44.894016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.894390 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.894391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:44.894454 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:44.894881 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:44.894886 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:44.895150 (+ 264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:44.896706 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:44.896707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:44.896708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:44.896876 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:44.897088 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:44.897089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:44.897089 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:44.897163 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:44.897475 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:44.897476 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:44.897520 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 06:52:44.897608 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 06:52:44.897704 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":59,"threads_started":1}
I20260926 06:52:44.897794 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:44.892329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:43051 (local address 127.0.191.254:44667)
0926 06:52:44.892373 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:44.892374 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:44.892429 (+ 55us) server_negotiation.cc:407] Connection header received
0926 06:52:44.892499 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:44.892499 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:44.892509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:44.892557 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:44.892916 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.893425 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.894097 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:44.894304 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:44.894388 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:44.895252 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:44.895253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:44.895254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:44.895260 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:44.896606 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:44.896960 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:44.896961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:44.896961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:44.897014 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:44.897229 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:44.897230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:44.897231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:44.897419 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:44.897482 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:44.897632 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 06:52:44.897703 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:52:44.899458 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 2091
I20260926 06:52:44.899526 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:52:44.899704 767 external_mini_cluster.cc:1428] Setting key 8cf16e039c579be2888809588ca702b5
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:52:44.924427 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:44667
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:37255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:45.002327 2231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:45.002516 2231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:45.002538 2231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:45.003643 2231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:45.003682 2231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:45.003705 2231 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:52:45.003726 2231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:52:45.005084 2231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:37255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.191.254:44667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:45.005247 2231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:45.005415 2231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:45.010188 2239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:45.010325 2231 server_base.cc:1034] running on GCE node
W20260926 06:52:45.010192 2237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:45.010205 2236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:45.010687 2231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:45.010874 2231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:45.012028 2231 hybrid_clock.cc:648] HybridClock initialized: now 1790405565012016 us; error 32 us; skew 500 ppm
Sep 26 06:52:45 dist-test-slave-ws74 krb5kdc[1834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405565, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:45.014469 2231 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:52:45.014887 2231 webserver.cc:460] Webserver started at http://127.0.191.195:45785/ using document root <none> and password file <none>
I20260926 06:52:45.015060 2231 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:45.015101 2231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:45.015185 2231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:45.016106 2231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "fda587220c88485bb5240769ffdb2d96"
format_stamp: "Formatted at 2026-09-26 06:52:45 on dist-test-slave-ws74"
server_key: "c2a629bdb0693067e7b3352a700aa8cd"
server_key_iv: "a9e7cabfeb6a85e85c8afe9307266bac"
server_key_version: "encryptionkey@0"
I20260926 06:52:45.016403 2231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "fda587220c88485bb5240769ffdb2d96"
format_stamp: "Formatted at 2026-09-26 06:52:45 on dist-test-slave-ws74"
server_key: "c2a629bdb0693067e7b3352a700aa8cd"
server_key_iv: "a9e7cabfeb6a85e85c8afe9307266bac"
server_key_version: "encryptionkey@0"
I20260926 06:52:45.017638 2231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:52:45.018302 2246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:45.018479 2231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:52:45.018546 2231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "fda587220c88485bb5240769ffdb2d96"
format_stamp: "Formatted at 2026-09-26 06:52:45 on dist-test-slave-ws74"
server_key: "c2a629bdb0693067e7b3352a700aa8cd"
server_key_iv: "a9e7cabfeb6a85e85c8afe9307266bac"
server_key_version: "encryptionkey@0"
I20260926 06:52:45.018644 2231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:45.035477 2231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:45.037300 2231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:45.037516 2231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:45.038036 2231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:45.038094 2231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:45.038134 2231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:45.038162 2231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 06:52:45 dist-test-slave-ws74 krb5kdc[1834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405565, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:45.044849 2231 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:34319
I20260926 06:52:45.044917 2361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:34319 every 8 connection(s)
I20260926 06:52:45.045185 2231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:52:45.047637 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.038446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:35089 (local address 127.0.191.254:44667)
0926 06:52:45.038533 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.038535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.038724 (+ 189us) server_negotiation.cc:407] Connection header received
0926 06:52:45.039161 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.039162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.039174 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.039257 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:45.039838 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.040670 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.042339 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.042549 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.042642 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.045007 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:45.045009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:45.045009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:45.045016 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:45.046409 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:45.046785 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:45.046786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:45.046786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:45.046832 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:45.047106 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:45.047107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:45.047107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:45.047231 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:45.047283 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.047484 (+ 201us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.047560 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 06:52:45.047896 2255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.038415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:44667 (local address 127.0.191.195:35089)
0926 06:52:45.038594 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:45.038640 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:45.039070 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:45.039302 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:45.039305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:45.039526 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:45.039747 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.039751 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.040776 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.040778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.042240 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.042243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.042661 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.042662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.042774 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.043287 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:45.043294 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:45.044923 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:45.046488 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:45.046494 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:45.046496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:45.046718 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:45.046889 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:45.046889 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:45.046889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:45.046976 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:45.047288 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:45.047292 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:45.047394 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 06:52:45.047471 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 06:52:45.047662 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":75,"threads_started":1}
I20260926 06:52:45.048403 2362 heartbeater.cc:347] Connected to a master server at 127.0.191.254:44667
I20260926 06:52:45.048496 2362 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:45.048695 2362 heartbeater.cc:511] Master 127.0.191.254:44667 requested a full tablet report, sending...
I20260926 06:52:45.048758 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 2231
I20260926 06:52:45.048856 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:52:45.049052 767 external_mini_cluster.cc:1428] Setting key e88c03979a431a4dcd991f005a2082e7
I20260926 06:52:45.049069 1878 ts_manager.cc:194] Registered new tserver with Master: fda587220c88485bb5240769ffdb2d96 (127.0.191.195:34319)
I20260926 06:52:45.049605 1878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:35089
I20260926 06:52:45.049825 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:52:45.054741 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.049425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:54881 (local address 127.0.191.254:44667)
0926 06:52:45.049460 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.049462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.049599 (+ 137us) server_negotiation.cc:407] Connection header received
0926 06:52:45.049646 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.049647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.049658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.049715 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:45.050058 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.050569 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.051263 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.051463 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.051544 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.052351 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:45.052352 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:45.052353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:45.052359 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:45.053662 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:45.054000 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:45.054001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:45.054002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:45.054051 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:45.054259 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:45.054260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:45.054261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:45.054367 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:45.054440 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.054585 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.054655 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:52:45.054761 2368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.049390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:44667 (local address 127.0.191.195:54881)
0926 06:52:45.049517 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:45.049537 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:45.049592 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:45.049761 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:45.049762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:45.049928 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:45.049973 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.049974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.050653 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.050654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.051165 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.051166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.051545 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.051546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.051608 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.052017 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:45.052021 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:45.052264 (+ 243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:45.053754 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:45.053755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:45.053756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:45.053918 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:45.054120 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:45.054121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:45.054121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:45.054189 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:45.054432 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:45.054432 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:45.054475 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 06:52:45.054562 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 06:52:45.054657 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
Sep 26 06:52:45 dist-test-slave-ws74 krb5kdc[1834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405565, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 06:52:45 dist-test-slave-ws74 krb5kdc[1834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405565, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:45.080649 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.073037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59464 (local address 127.0.191.254:44667)
0926 06:52:45.073079 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.073081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.073090 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:52:45.073115 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.073115 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.073127 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.073193 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:45.074013 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.074800 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.075853 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.076166 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.076233 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.078752 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:45.078753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:45.078753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:45.078759 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:45.079814 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:45.080140 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:45.080141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:45.080142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:45.080188 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:45.080302 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:45.080302 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:45.080302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:45.080385 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:45.080454 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.080533 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.080585 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:52:45.082180 1879 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:52:45.083711 1879 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:52:45.090164 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.086051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:40009 (local address 127.0.0.1:39902)
0926 06:52:45.086203 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:45.086212 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:45.086270 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:45.086568 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:45.086570 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:45.086572 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:45.086635 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.086637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.087899 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.087899 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.089011 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.089013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.089858 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.089859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.089950 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.089964 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 06:52:45.090035 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 06:52:45.090089 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":2,"thread_start_us":41,"threads_started":1}
I20260926 06:52:45.090180 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.086173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39902 (local address 127.0.191.193:40009)
0926 06:52:45.086352 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.086356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.086367 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:52:45.086407 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.086409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.086425 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.086502 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:45.086719 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.087821 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.089109 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.089759 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.089856 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.089934 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.090057 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.090093 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260926 06:52:45.090830 2016 tablet_service.cc:1467] Processing CreateTablet for tablet 18cb103c8f4748d7a39a6c14fb5fd04d (DEFAULT_TABLE table=test-table [id=14cc8b626dfd4774849457ac2b2b2139]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:45.091245 2016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18cb103c8f4748d7a39a6c14fb5fd04d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:45.091460 2377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.085879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:34319 (local address 127.0.0.1:33644)
0926 06:52:45.086157 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:45.086195 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:45.086270 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:45.086596 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:45.086597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:45.086599 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:45.086640 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.086642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.089102 (+ 2460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.089103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.090174 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.090176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.091218 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.091219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.091279 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.091283 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:52:45.091336 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 06:52:45.091373 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260926 06:52:45.091708 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.085879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:40059 (local address 127.0.0.1:50236)
0926 06:52:45.086154 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:45.086198 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:45.086270 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:45.086586 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:45.086586 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:45.086588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:45.086641 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.086642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.088938 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.088939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.090123 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.090125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.091480 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.091481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.091542 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.091546 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:52:45.091595 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 06:52:45.091642 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":47,"threads_started":1}
I20260926 06:52:45.091763 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.085976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33644 (local address 127.0.191.195:34319)
0926 06:52:45.086395 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.086399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.086408 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:52:45.086444 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.086446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.086461 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.086533 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:45.086755 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.087822 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.090259 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.091013 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.091083 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.091183 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.091624 (+ 441us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.091666 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":78,"threads_started":1}
I20260926 06:52:45.092077 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.085989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50236 (local address 127.0.191.194:40059)
0926 06:52:45.086163 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.086168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.086390 (+ 222us) server_negotiation.cc:407] Connection header received
0926 06:52:45.086432 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.086434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.086450 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.086502 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:45.087930 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.088848 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.090489 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.091405 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.091470 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.091744 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.091943 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.091981 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":77,"threads_started":1}
I20260926 06:52:45.092455 2296 tablet_service.cc:1467] Processing CreateTablet for tablet 18cb103c8f4748d7a39a6c14fb5fd04d (DEFAULT_TABLE table=test-table [id=14cc8b626dfd4774849457ac2b2b2139]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:45.092618 2156 tablet_service.cc:1467] Processing CreateTablet for tablet 18cb103c8f4748d7a39a6c14fb5fd04d (DEFAULT_TABLE table=test-table [id=14cc8b626dfd4774849457ac2b2b2139]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:45.092777 2296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18cb103c8f4748d7a39a6c14fb5fd04d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:45.092907 2156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18cb103c8f4748d7a39a6c14fb5fd04d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:45.095839 2382 tablet_bootstrap.cc:492] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e: Bootstrap starting.
I20260926 06:52:45.096437 2382 tablet_bootstrap.cc:654] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:45.096411 2383 tablet_bootstrap.cc:492] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d: Bootstrap starting.
I20260926 06:52:45.096673 2384 tablet_bootstrap.cc:492] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96: Bootstrap starting.
I20260926 06:52:45.096786 2382 log.cc:824] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:45.097378 2384 tablet_bootstrap.cc:654] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:45.097687 2382 tablet_bootstrap.cc:492] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e: No bootstrap required, opened a new log
I20260926 06:52:45.097756 2382 ts_tablet_manager.cc:1397] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:52:45.097756 2384 log.cc:824] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:45.097995 2383 tablet_bootstrap.cc:654] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:45.098320 2383 log.cc:824] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:45.098551 2384 tablet_bootstrap.cc:492] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96: No bootstrap required, opened a new log
I20260926 06:52:45.098618 2384 ts_tablet_manager.cc:1397] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:52:45.098949 2383 tablet_bootstrap.cc:492] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d: No bootstrap required, opened a new log
I20260926 06:52:45.099030 2383 ts_tablet_manager.cc:1397] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 06:52:45.099728 2382 raft_consensus.cc:348] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } }
I20260926 06:52:45.099893 2382 raft_consensus.cc:374] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:45.099929 2382 raft_consensus.cc:729] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f6eafd243c466a946da5e79675845e, State: Initialized, Role: FOLLOWER
I20260926 06:52:45.100065 2382 consensus_queue.cc:260] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } }
I20260926 06:52:45.100324 2382 ts_tablet_manager.cc:1428] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:52:45.100443 2082 heartbeater.cc:503] Master 127.0.191.254:44667 was elected leader, sending a full tablet report...
I20260926 06:52:45.100593 2384 raft_consensus.cc:348] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } }
I20260926 06:52:45.100754 2384 raft_consensus.cc:374] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:45.100651 2383 raft_consensus.cc:348] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } }
I20260926 06:52:45.100796 2384 raft_consensus.cc:729] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fda587220c88485bb5240769ffdb2d96, State: Initialized, Role: FOLLOWER
I20260926 06:52:45.100805 2383 raft_consensus.cc:374] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:45.100841 2383 raft_consensus.cc:729] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73bc817f2231454c8eb7a1545535452d, State: Initialized, Role: FOLLOWER
I20260926 06:52:45.100910 2384 consensus_queue.cc:260] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } }
I20260926 06:52:45.100970 2383 consensus_queue.cc:260] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } }
I20260926 06:52:45.101158 2384 ts_tablet_manager.cc:1428] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:52:45.101221 2362 heartbeater.cc:503] Master 127.0.191.254:44667 was elected leader, sending a full tablet report...
I20260926 06:52:45.101689 2383 ts_tablet_manager.cc:1428] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 06:52:45.101812 2222 heartbeater.cc:503] Master 127.0.191.254:44667 was elected leader, sending a full tablet report...
W20260926 06:52:45.140046 2223 tablet.cc:2369] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:52:45.186525 2389 raft_consensus.cc:482] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:52:45.186697 2389 raft_consensus.cc:504] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } }
I20260926 06:52:45.187084 2389 leader_election.cc:288] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73bc817f2231454c8eb7a1545535452d (127.0.191.194:40059), 20f6eafd243c466a946da5e79675845e (127.0.191.193:40009)
I20260926 06:52:45.190220 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.187563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:35133 (local address 127.0.191.194:40059)
0926 06:52:45.187601 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.187603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.187612 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:52:45.187659 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.187660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.187669 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.187727 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:45.187930 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.188506 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.189417 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.189841 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.189948 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.189960 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.190101 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.190145 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:52:45.190235 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.187487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:57017 (local address 127.0.191.193:40009)
0926 06:52:45.187532 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.187535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.187544 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:52:45.187577 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.187578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.187589 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.187644 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:45.187837 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.188343 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.189477 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.189835 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.189893 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.189900 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.190136 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.190175 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:52:45.190235 2391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.187378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:40059 (local address 127.0.191.195:35133)
0926 06:52:45.187547 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:45.187556 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:45.187609 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:45.187760 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:45.187760 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:45.187762 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:45.187855 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.187856 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.188594 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.188595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.189304 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.189306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.189970 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.189971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.190022 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.190025 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:52:45.190068 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 06:52:45.190114 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":15,"thread_start_us":93,"threads_started":1}
I20260926 06:52:45.190243 2255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.187378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:40009 (local address 127.0.191.195:57017)
0926 06:52:45.187443 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:45.187458 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:45.187515 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:45.187653 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:45.187653 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:45.187655 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:45.187747 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.187750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.188427 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.188428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.189309 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.189311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.189928 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.189929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.190034 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.190037 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:52:45.190123 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 06:52:45.190161 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 06:52:45.190578 2176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18cb103c8f4748d7a39a6c14fb5fd04d" candidate_uuid: "fda587220c88485bb5240769ffdb2d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73bc817f2231454c8eb7a1545535452d" is_pre_election: true
I20260926 06:52:45.190749 2176 raft_consensus.cc:2454] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fda587220c88485bb5240769ffdb2d96 in term 0.
I20260926 06:52:45.190872 2036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18cb103c8f4748d7a39a6c14fb5fd04d" candidate_uuid: "fda587220c88485bb5240769ffdb2d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f6eafd243c466a946da5e79675845e" is_pre_election: true
I20260926 06:52:45.190991 2249 leader_election.cc:302] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73bc817f2231454c8eb7a1545535452d, fda587220c88485bb5240769ffdb2d96; no voters:
I20260926 06:52:45.191048 2036 raft_consensus.cc:2454] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fda587220c88485bb5240769ffdb2d96 in term 0.
I20260926 06:52:45.191135 2389 raft_consensus.cc:2781] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:52:45.191205 2389 raft_consensus.cc:482] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:52:45.191242 2389 raft_consensus.cc:3037] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:45.192278 2389 raft_consensus.cc:504] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } }
I20260926 06:52:45.192443 2389 leader_election.cc:288] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [CANDIDATE]: Term 1 election: Requested vote from peers 73bc817f2231454c8eb7a1545535452d (127.0.191.194:40059), 20f6eafd243c466a946da5e79675845e (127.0.191.193:40009)
I20260926 06:52:45.192584 2176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18cb103c8f4748d7a39a6c14fb5fd04d" candidate_uuid: "fda587220c88485bb5240769ffdb2d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73bc817f2231454c8eb7a1545535452d"
I20260926 06:52:45.192600 2036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18cb103c8f4748d7a39a6c14fb5fd04d" candidate_uuid: "fda587220c88485bb5240769ffdb2d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f6eafd243c466a946da5e79675845e"
I20260926 06:52:45.192668 2036 raft_consensus.cc:3037] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:45.192668 2176 raft_consensus.cc:3037] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:45.193605 2036 raft_consensus.cc:2454] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fda587220c88485bb5240769ffdb2d96 in term 1.
I20260926 06:52:45.193605 2176 raft_consensus.cc:2454] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fda587220c88485bb5240769ffdb2d96 in term 1.
I20260926 06:52:45.193796 2248 leader_election.cc:302] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f6eafd243c466a946da5e79675845e, fda587220c88485bb5240769ffdb2d96; no voters:
I20260926 06:52:45.193897 2389 raft_consensus.cc:2781] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:45.194041 2389 raft_consensus.cc:686] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [term 1 LEADER]: Becoming Leader. State: Replica: fda587220c88485bb5240769ffdb2d96, State: Running, Role: LEADER
I20260926 06:52:45.194113 2389 consensus_queue.cc:237] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } }
I20260926 06:52:45.195022 1879 catalog_manager.cc:5495] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 reported cstate change: term changed from 0 to 1, leader changed from <none> to fda587220c88485bb5240769ffdb2d96 (127.0.191.195). New cstate: current_term: 1 leader_uuid: "fda587220c88485bb5240769ffdb2d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } health_report { overall_health: UNKNOWN } } }
I20260926 06:52:45.205191 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.202959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33660 (local address 127.0.191.195:34319)
0926 06:52:45.203073 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.203075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.203082 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:52:45.203108 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.203108 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.203123 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.203182 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:52:45.203299 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.203769 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.204463 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.204654 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.204707 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.204870 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:52:45.204964 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:52:45.205063 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.205080 (+ 17us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.205097 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260926 06:52:45.206760 2176 raft_consensus.cc:1261] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d [term 1 FOLLOWER]: Refusing update from remote peer fda587220c88485bb5240769ffdb2d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:45.206822 2036 raft_consensus.cc:1261] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e [term 1 FOLLOWER]: Refusing update from remote peer fda587220c88485bb5240769ffdb2d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:45.207178 2389 consensus_queue.cc:1035] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:45.207341 2392 consensus_queue.cc:1035] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:45.209779 2395 mvcc.cc:205] Tried to move back new op lower bound from 7333501195084652544 to 7333501195036086272. Current Snapshot: MvccSnapshot[applied={T|T < 7333501195084652544}]
I20260926 06:52:45.210067 2396 mvcc.cc:205] Tried to move back new op lower bound from 7333501195084652544 to 7333501195036086272. Current Snapshot: MvccSnapshot[applied={T|T < 7333501195084652544}]
I20260926 06:52:45.213294 1878 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:52:45.213465 1878 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:52:45.214347 1878 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874: Sending DeleteTablet for 3 replicas of tablet 18cb103c8f4748d7a39a6c14fb5fd04d
I20260926 06:52:45.214752 2016 tablet_service.cc:1514] Processing DeleteTablet for tablet 18cb103c8f4748d7a39a6c14fb5fd04d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:45 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:39902
I20260926 06:52:45.214787 2296 tablet_service.cc:1514] Processing DeleteTablet for tablet 18cb103c8f4748d7a39a6c14fb5fd04d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:45 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:33644
I20260926 06:52:45.214819 2156 tablet_service.cc:1514] Processing DeleteTablet for tablet 18cb103c8f4748d7a39a6c14fb5fd04d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:45 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:50236
I20260926 06:52:45.215020 2404 tablet_replica.cc:330] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e: stopping tablet replica
I20260926 06:52:45.215142 2404 raft_consensus.cc:2229] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:52:45.215257 2404 raft_consensus.cc:2258] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:45.215202 1878 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:52:45.215323 1878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:52:45.215692 2404 ts_tablet_manager.cc:1905] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:45.216823 2156 tablet_service.cc:1467] Processing CreateTablet for tablet f221cc9f618d4fbb8074da4748ef7473 (DEFAULT_TABLE table=test-table [id=09ae2f954d2e4ca09e5f256f976d7b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:45.216878 2016 tablet_service.cc:1467] Processing CreateTablet for tablet f221cc9f618d4fbb8074da4748ef7473 (DEFAULT_TABLE table=test-table [id=09ae2f954d2e4ca09e5f256f976d7b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:45.216934 2156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f221cc9f618d4fbb8074da4748ef7473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:45.217033 2016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f221cc9f618d4fbb8074da4748ef7473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:45.217159 2405 tablet_replica.cc:330] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96: stopping tablet replica
I20260926 06:52:45.217267 2405 raft_consensus.cc:2229] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:52:45.217340 2405 raft_consensus.cc:2258] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:45.217478 2296 tablet_service.cc:1467] Processing CreateTablet for tablet f221cc9f618d4fbb8074da4748ef7473 (DEFAULT_TABLE table=test-table [id=09ae2f954d2e4ca09e5f256f976d7b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:45.217567 2296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f221cc9f618d4fbb8074da4748ef7473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:45.218046 2404 ts_tablet_manager.cc:1918] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:52:45.218102 2404 log.cc:1196] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/wal/wals/18cb103c8f4748d7a39a6c14fb5fd04d
I20260926 06:52:45.218241 2383 tablet_bootstrap.cc:492] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d: Bootstrap starting.
I20260926 06:52:45.218403 2404 ts_tablet_manager.cc:1939] T 18cb103c8f4748d7a39a6c14fb5fd04d P 20f6eafd243c466a946da5e79675845e: Deleting consensus metadata
I20260926 06:52:45.218633 2383 tablet_bootstrap.cc:654] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:45.218856 1864 catalog_manager.cc:4841] TS 20f6eafd243c466a946da5e79675845e (127.0.191.193:40009): tablet 18cb103c8f4748d7a39a6c14fb5fd04d (table test-table [id=14cc8b626dfd4774849457ac2b2b2139]) successfully deleted
I20260926 06:52:45.219036 2384 tablet_bootstrap.cc:492] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96: Bootstrap starting.
I20260926 06:52:45.219341 2405 ts_tablet_manager.cc:1905] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:45.219429 2384 tablet_bootstrap.cc:654] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:45.220980 2382 tablet_bootstrap.cc:492] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e: Bootstrap starting.
I20260926 06:52:45.221025 2405 ts_tablet_manager.cc:1918] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:52:45.221181 2405 log.cc:1196] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/18cb103c8f4748d7a39a6c14fb5fd04d
I20260926 06:52:45.215718 2406 tablet_replica.cc:330] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d: stopping tablet replica
I20260926 06:52:45.221222 2384 tablet_bootstrap.cc:492] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96: No bootstrap required, opened a new log
I20260926 06:52:45.221426 2382 tablet_bootstrap.cc:654] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:45.221426 2384 ts_tablet_manager.cc:1397] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:52:45.221588 2384 raft_consensus.cc:348] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.221697 2384 raft_consensus.cc:374] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:45.221726 2384 raft_consensus.cc:729] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fda587220c88485bb5240769ffdb2d96, State: Initialized, Role: FOLLOWER
I20260926 06:52:45.221771 2384 consensus_queue.cc:260] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.221860 2384 ts_tablet_manager.cc:1428] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:45.221921 2405 ts_tablet_manager.cc:1939] T 18cb103c8f4748d7a39a6c14fb5fd04d P fda587220c88485bb5240769ffdb2d96: Deleting consensus metadata
I20260926 06:52:45.221342 2406 raft_consensus.cc:2229] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:52:45.222090 2406 raft_consensus.cc:2258] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:45.222546 2406 ts_tablet_manager.cc:1905] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:45.222851 1864 catalog_manager.cc:4841] TS fda587220c88485bb5240769ffdb2d96 (127.0.191.195:34319): tablet 18cb103c8f4748d7a39a6c14fb5fd04d (table test-table [id=14cc8b626dfd4774849457ac2b2b2139]) successfully deleted
I20260926 06:52:45.224087 2382 tablet_bootstrap.cc:492] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e: No bootstrap required, opened a new log
I20260926 06:52:45.224135 2382 ts_tablet_manager.cc:1397] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260926 06:52:45.224308 2406 ts_tablet_manager.cc:1918] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:52:45.224309 2382 raft_consensus.cc:348] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.224375 2406 log.cc:1196] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/18cb103c8f4748d7a39a6c14fb5fd04d
I20260926 06:52:45.224397 2382 raft_consensus.cc:374] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:45.224431 2382 raft_consensus.cc:729] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f6eafd243c466a946da5e79675845e, State: Initialized, Role: FOLLOWER
I20260926 06:52:45.224470 2382 consensus_queue.cc:260] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.224579 2382 ts_tablet_manager.cc:1428] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 06:52:45.224699 2406 ts_tablet_manager.cc:1939] T 18cb103c8f4748d7a39a6c14fb5fd04d P 73bc817f2231454c8eb7a1545535452d: Deleting consensus metadata
I20260926 06:52:45.225044 1865 catalog_manager.cc:4841] TS 73bc817f2231454c8eb7a1545535452d (127.0.191.194:40059): tablet 18cb103c8f4748d7a39a6c14fb5fd04d (table test-table [id=14cc8b626dfd4774849457ac2b2b2139]) successfully deleted
I20260926 06:52:45.225765 2383 tablet_bootstrap.cc:492] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d: No bootstrap required, opened a new log
I20260926 06:52:45.225817 2383 ts_tablet_manager.cc:1397] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260926 06:52:45.225946 2383 raft_consensus.cc:348] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.226011 2383 raft_consensus.cc:374] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:45.226032 2383 raft_consensus.cc:729] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73bc817f2231454c8eb7a1545535452d, State: Initialized, Role: FOLLOWER
I20260926 06:52:45.226073 2383 consensus_queue.cc:260] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.226163 2383 ts_tablet_manager.cc:1428] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260926 06:52:45.244784 2083 tablet.cc:2369] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:52:45.295680 2363 tablet.cc:2369] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:52:45.388358 2389 raft_consensus.cc:482] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:52:45.388492 2389 raft_consensus.cc:504] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.388705 2389 leader_election.cc:288] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f6eafd243c466a946da5e79675845e (127.0.191.193:40009), 73bc817f2231454c8eb7a1545535452d (127.0.191.194:40059)
I20260926 06:52:45.388904 2176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f221cc9f618d4fbb8074da4748ef7473" candidate_uuid: "fda587220c88485bb5240769ffdb2d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73bc817f2231454c8eb7a1545535452d" is_pre_election: true
I20260926 06:52:45.388936 2036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f221cc9f618d4fbb8074da4748ef7473" candidate_uuid: "fda587220c88485bb5240769ffdb2d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f6eafd243c466a946da5e79675845e" is_pre_election: true
I20260926 06:52:45.389006 2176 raft_consensus.cc:2454] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fda587220c88485bb5240769ffdb2d96 in term 0.
I20260926 06:52:45.389052 2036 raft_consensus.cc:2454] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fda587220c88485bb5240769ffdb2d96 in term 0.
I20260926 06:52:45.389187 2249 leader_election.cc:302] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73bc817f2231454c8eb7a1545535452d, fda587220c88485bb5240769ffdb2d96; no voters:
I20260926 06:52:45.389307 2389 raft_consensus.cc:2781] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:52:45.389360 2389 raft_consensus.cc:482] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:52:45.389392 2389 raft_consensus.cc:3037] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:45.390038 2389 raft_consensus.cc:504] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.390174 2389 leader_election.cc:288] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [CANDIDATE]: Term 1 election: Requested vote from peers 20f6eafd243c466a946da5e79675845e (127.0.191.193:40009), 73bc817f2231454c8eb7a1545535452d (127.0.191.194:40059)
I20260926 06:52:45.390316 2036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f221cc9f618d4fbb8074da4748ef7473" candidate_uuid: "fda587220c88485bb5240769ffdb2d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f6eafd243c466a946da5e79675845e"
I20260926 06:52:45.390365 2176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f221cc9f618d4fbb8074da4748ef7473" candidate_uuid: "fda587220c88485bb5240769ffdb2d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73bc817f2231454c8eb7a1545535452d"
I20260926 06:52:45.390427 2036 raft_consensus.cc:3037] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:45.390452 2176 raft_consensus.cc:3037] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:45.391144 2036 raft_consensus.cc:2454] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fda587220c88485bb5240769ffdb2d96 in term 1.
I20260926 06:52:45.391165 2176 raft_consensus.cc:2454] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fda587220c88485bb5240769ffdb2d96 in term 1.
I20260926 06:52:45.391319 2248 leader_election.cc:302] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f6eafd243c466a946da5e79675845e, fda587220c88485bb5240769ffdb2d96; no voters:
I20260926 06:52:45.391422 2389 raft_consensus.cc:2781] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:45.391479 2389 raft_consensus.cc:686] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [term 1 LEADER]: Becoming Leader. State: Replica: fda587220c88485bb5240769ffdb2d96, State: Running, Role: LEADER
I20260926 06:52:45.391527 2389 consensus_queue.cc:237] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.392138 1878 catalog_manager.cc:5495] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 reported cstate change: term changed from 0 to 1, leader changed from <none> to fda587220c88485bb5240769ffdb2d96 (127.0.191.195). New cstate: current_term: 1 leader_uuid: "fda587220c88485bb5240769ffdb2d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } health_report { overall_health: HEALTHY } } }
Sep 26 06:52:45 dist-test-slave-ws74 krb5kdc[1834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405564, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:45.424654 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.417954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59472 (local address 127.0.191.254:44667)
0926 06:52:45.418009 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.418011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.418150 (+ 139us) server_negotiation.cc:407] Connection header received
0926 06:52:45.418195 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.418196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.418203 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.418266 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:45.418754 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.419265 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.419930 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.420103 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.420171 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.422202 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:45.422203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:45.422204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:45.422210 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:45.423414 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:45.423810 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:45.423812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:45.423812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:45.423869 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:45.424085 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:45.424086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:45.424086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:45.424203 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:45.424274 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.424419 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.424544 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":30}
I20260926 06:52:45.424733 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.417916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:44667 (local address 127.0.0.1:59472)
0926 06:52:45.418078 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:45.418090 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:45.418146 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:45.418320 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:45.418321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:45.418645 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:45.418687 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.418689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.419335 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.419336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.419858 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.419860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.420181 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.420182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.420220 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.420670 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:45.420675 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:45.422097 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:45.423509 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:45.423516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:45.423518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:45.423718 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:45.423944 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:45.423945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:45.423945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:45.424011 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:45.424265 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:45.424271 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:45.424313 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 06:52:45.424397 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 06:52:45.424612 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":91,"thread_start_us":83,"threads_started":1}
I20260926 06:52:45.426525 1878 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:59472:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 06:52:45.426749 1878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:52:45.428045 2156 tablet_service.cc:1467] Processing CreateTablet for tablet 31ca266582164014a09cea8afe329ba1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff4dde45a2c94c4f8ba809d05dd84c2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:52:45.428071 2296 tablet_service.cc:1467] Processing CreateTablet for tablet 31ca266582164014a09cea8afe329ba1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff4dde45a2c94c4f8ba809d05dd84c2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:52:45.428112 2016 tablet_service.cc:1467] Processing CreateTablet for tablet 31ca266582164014a09cea8afe329ba1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff4dde45a2c94c4f8ba809d05dd84c2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:52:45.428201 2156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ca266582164014a09cea8afe329ba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:45.428220 2296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ca266582164014a09cea8afe329ba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:45.428231 2016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ca266582164014a09cea8afe329ba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:45.429265 2384 tablet_bootstrap.cc:492] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96: Bootstrap starting.
I20260926 06:52:45.429576 2384 tablet_bootstrap.cc:654] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:45.429662 2382 tablet_bootstrap.cc:492] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e: Bootstrap starting.
I20260926 06:52:45.429677 2383 tablet_bootstrap.cc:492] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d: Bootstrap starting.
I20260926 06:52:45.430071 2383 tablet_bootstrap.cc:654] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:45.430076 2382 tablet_bootstrap.cc:654] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:45.430220 2384 tablet_bootstrap.cc:492] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96: No bootstrap required, opened a new log
I20260926 06:52:45.430265 2384 ts_tablet_manager.cc:1397] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:52:45.430382 2384 raft_consensus.cc:348] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.430450 2384 raft_consensus.cc:374] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:45.430476 2384 raft_consensus.cc:729] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fda587220c88485bb5240769ffdb2d96, State: Initialized, Role: FOLLOWER
I20260926 06:52:45.430523 2384 consensus_queue.cc:260] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.430692 2383 tablet_bootstrap.cc:492] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d: No bootstrap required, opened a new log
I20260926 06:52:45.430729 2383 ts_tablet_manager.cc:1397] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:52:45.430644 2392 tablet_replica.cc:439] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } }
I20260926 06:52:45.430648 2384 ts_tablet_manager.cc:1428] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:45.430786 2392 tablet_replica.cc:442] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:52:45.430871 2383 raft_consensus.cc:348] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.430944 2383 raft_consensus.cc:374] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:45.430965 2383 raft_consensus.cc:729] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73bc817f2231454c8eb7a1545535452d, State: Initialized, Role: FOLLOWER
I20260926 06:52:45.431005 2382 tablet_bootstrap.cc:492] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e: No bootstrap required, opened a new log
I20260926 06:52:45.431026 2383 consensus_queue.cc:260] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.431056 2382 ts_tablet_manager.cc:1397] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 06:52:45.431137 2383 ts_tablet_manager.cc:1428] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:45.431195 2382 raft_consensus.cc:348] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.431181 2390 tablet_replica.cc:439] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } }
I20260926 06:52:45.431265 2382 raft_consensus.cc:374] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:45.431291 2382 raft_consensus.cc:729] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f6eafd243c466a946da5e79675845e, State: Initialized, Role: FOLLOWER
I20260926 06:52:45.431288 2390 tablet_replica.cc:442] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:52:45.431336 2382 consensus_queue.cc:260] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.431416 2382 ts_tablet_manager.cc:1428] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 06:52:45.431589 2388 tablet_replica.cc:439] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } }
I20260926 06:52:45.431712 2388 tablet_replica.cc:442] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:52:45.470314 2390 raft_consensus.cc:482] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:52:45.470392 2390 raft_consensus.cc:504] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.470713 2390 leader_election.cc:288] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f6eafd243c466a946da5e79675845e (127.0.191.193:40009), fda587220c88485bb5240769ffdb2d96 (127.0.191.195:34319)
I20260926 06:52:45.471621 2392 raft_consensus.cc:482] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:52:45.471691 2392 raft_consensus.cc:504] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.471812 2392 leader_election.cc:288] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f6eafd243c466a946da5e79675845e (127.0.191.193:40009), 73bc817f2231454c8eb7a1545535452d (127.0.191.194:40059)
I20260926 06:52:45.471948 2036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ca266582164014a09cea8afe329ba1" candidate_uuid: "fda587220c88485bb5240769ffdb2d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f6eafd243c466a946da5e79675845e" is_pre_election: true
I20260926 06:52:45.472020 2036 raft_consensus.cc:2454] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fda587220c88485bb5240769ffdb2d96 in term 0.
I20260926 06:52:45.472010 2176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ca266582164014a09cea8afe329ba1" candidate_uuid: "fda587220c88485bb5240769ffdb2d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73bc817f2231454c8eb7a1545535452d" is_pre_election: true
I20260926 06:52:45.472082 2176 raft_consensus.cc:2454] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fda587220c88485bb5240769ffdb2d96 in term 0.
I20260926 06:52:45.472141 2248 leader_election.cc:302] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f6eafd243c466a946da5e79675845e, fda587220c88485bb5240769ffdb2d96; no voters:
I20260926 06:52:45.472218 2392 raft_consensus.cc:2781] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:52:45.472250 2392 raft_consensus.cc:482] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:52:45.472275 2392 raft_consensus.cc:3037] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:45.473542 2392 raft_consensus.cc:504] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.473662 2392 leader_election.cc:288] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [CANDIDATE]: Term 1 election: Requested vote from peers 20f6eafd243c466a946da5e79675845e (127.0.191.193:40009), 73bc817f2231454c8eb7a1545535452d (127.0.191.194:40059)
I20260926 06:52:45.473806 2036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ca266582164014a09cea8afe329ba1" candidate_uuid: "fda587220c88485bb5240769ffdb2d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f6eafd243c466a946da5e79675845e"
I20260926 06:52:45.473865 2036 raft_consensus.cc:3037] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:45.473838 2176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ca266582164014a09cea8afe329ba1" candidate_uuid: "fda587220c88485bb5240769ffdb2d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73bc817f2231454c8eb7a1545535452d"
I20260926 06:52:45.473896 2176 raft_consensus.cc:3037] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:45.473971 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.470973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:59289 (local address 127.0.191.193:40009)
0926 06:52:45.471039 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.471041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.471103 (+ 62us) server_negotiation.cc:407] Connection header received
0926 06:52:45.471174 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.471175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.471184 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.471239 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:45.471391 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.472079 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.473145 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.473586 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.473683 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.473692 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.473847 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.473889 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 06:52:45.474222 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.470879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:40009 (local address 127.0.191.194:59289)
0926 06:52:45.471049 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:45.471060 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:45.471121 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:45.471247 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:45.471247 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:45.471249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:45.471311 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.471313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.472176 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.472177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.473047 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.473049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.473683 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.473684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.473759 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.473765 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 06:52:45.474094 (+ 329us) client_negotiation.cc:240] Negotiation successful
0926 06:52:45.474146 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":72,"thread_start_us":90,"threads_started":1}
I20260926 06:52:45.474538 2176 raft_consensus.cc:2454] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fda587220c88485bb5240769ffdb2d96 in term 1.
I20260926 06:52:45.474607 2036 raft_consensus.cc:2454] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fda587220c88485bb5240769ffdb2d96 in term 1.
I20260926 06:52:45.474709 2249 leader_election.cc:302] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73bc817f2231454c8eb7a1545535452d, fda587220c88485bb5240769ffdb2d96; no voters:
I20260926 06:52:45.474812 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.471153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:54867 (local address 127.0.191.195:34319)
0926 06:52:45.471195 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.471198 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.471253 (+ 55us) server_negotiation.cc:407] Connection header received
0926 06:52:45.471326 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.471327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.471344 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.471404 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:45.471550 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.472174 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.473948 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.474408 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.474454 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.474461 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.474696 (+ 235us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.474751 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:52:45.474869 2036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ca266582164014a09cea8afe329ba1" candidate_uuid: "73bc817f2231454c8eb7a1545535452d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f6eafd243c466a946da5e79675845e" is_pre_election: true
I20260926 06:52:45.474943 2036 raft_consensus.cc:2379] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73bc817f2231454c8eb7a1545535452d in current term 1: Already voted for candidate fda587220c88485bb5240769ffdb2d96 in this term.
I20260926 06:52:45.475015 2421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.471090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:34319 (local address 127.0.191.194:54867)
0926 06:52:45.471189 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:45.471198 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:45.471264 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:45.471407 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:45.471408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:45.471409 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:45.471469 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.471471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.472260 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.472261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.473853 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.473855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.474520 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.474521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.474605 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.474608 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:52:45.474899 (+ 291us) client_negotiation.cc:240] Negotiation successful
0926 06:52:45.474937 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":43,"threads_started":1}
I20260926 06:52:45.475030 2392 raft_consensus.cc:2781] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:45.475078 2392 raft_consensus.cc:686] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [term 1 LEADER]: Becoming Leader. State: Replica: fda587220c88485bb5240769ffdb2d96, State: Running, Role: LEADER
I20260926 06:52:45.475118 2392 consensus_queue.cc:237] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } }
I20260926 06:52:45.475332 2316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ca266582164014a09cea8afe329ba1" candidate_uuid: "73bc817f2231454c8eb7a1545535452d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fda587220c88485bb5240769ffdb2d96" is_pre_election: true
I20260926 06:52:45.475389 2389 tablet_replica.cc:439] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96: TxnStatusTablet state changed. Reason: New leader fda587220c88485bb5240769ffdb2d96. Latest consensus state: current_term: 1 leader_uuid: "fda587220c88485bb5240769ffdb2d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } }
I20260926 06:52:45.475442 2389 tablet_replica.cc:442] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:52:45.475585 2423 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 06:52:45.475642 2108 leader_election.cc:302] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73bc817f2231454c8eb7a1545535452d; no voters: 20f6eafd243c466a946da5e79675845e, fda587220c88485bb5240769ffdb2d96
I20260926 06:52:45.475695 2423 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 06:52:45.475767 1878 catalog_manager.cc:5495] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 reported cstate change: term changed from 0 to 1, leader changed from <none> to fda587220c88485bb5240769ffdb2d96 (127.0.191.195). New cstate: current_term: 1 leader_uuid: "fda587220c88485bb5240769ffdb2d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } health_report { overall_health: HEALTHY } } }
I20260926 06:52:45.476025 2390 raft_consensus.cc:2726] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 06:52:45.490952 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.488698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33668 (local address 127.0.191.195:34319)
0926 06:52:45.488745 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.488747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.488756 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:52:45.488830 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.488831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.488840 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.488878 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:52:45.488999 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.489506 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.490228 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.490432 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.490515 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.490716 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:52:45.490748 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:52:45.490798 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.490871 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.490889 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 06:52:45.490983 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.488632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:34319 (local address 127.0.0.1:33668)
0926 06:52:45.488703 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:45.488715 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:45.488762 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:45.488894 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:45.488895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:45.488896 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:52:45.488937 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.488939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.489585 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.489586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.490146 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.490148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.490518 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.490519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.490584 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.490637 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:52:45.490811 (+ 174us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:52:45.490817 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 06:52:45.490867 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 06:52:45.490907 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 06:52:45.491792 2176 raft_consensus.cc:1261] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d [term 1 FOLLOWER]: Refusing update from remote peer fda587220c88485bb5240769ffdb2d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:45.491812 2036 raft_consensus.cc:1261] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e [term 1 FOLLOWER]: Refusing update from remote peer fda587220c88485bb5240769ffdb2d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:45.491971 2389 consensus_queue.cc:1035] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:45.492048 2392 consensus_queue.cc:1035] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:45.492550 2390 tablet_replica.cc:439] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d: TxnStatusTablet state changed. Reason: New leader fda587220c88485bb5240769ffdb2d96. Latest consensus state: current_term: 1 leader_uuid: "fda587220c88485bb5240769ffdb2d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } }
I20260926 06:52:45.492647 2390 tablet_replica.cc:442] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:52:45.492599 2388 tablet_replica.cc:439] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e: TxnStatusTablet state changed. Reason: New leader fda587220c88485bb5240769ffdb2d96. Latest consensus state: current_term: 1 leader_uuid: "fda587220c88485bb5240769ffdb2d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } }
I20260926 06:52:45.492691 2388 tablet_replica.cc:442] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:52:45.493342 2392 tablet_replica.cc:439] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fda587220c88485bb5240769ffdb2d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } }
I20260926 06:52:45.493408 2428 tablet_replica.cc:439] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fda587220c88485bb5240769ffdb2d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } }
I20260926 06:52:45.493458 2392 tablet_replica.cc:442] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:52:45.493467 2428 tablet_replica.cc:442] T 31ca266582164014a09cea8afe329ba1 P fda587220c88485bb5240769ffdb2d96: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:52:45.493651 2397 mvcc.cc:205] Tried to move back new op lower bound from 7333501196252983296 to 7333501196186423296. Current Snapshot: MvccSnapshot[applied={T|T < 7333501196252983296}]
I20260926 06:52:45.493824 2388 tablet_replica.cc:439] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fda587220c88485bb5240769ffdb2d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } }
I20260926 06:52:45.493896 2388 tablet_replica.cc:442] T 31ca266582164014a09cea8afe329ba1 P 20f6eafd243c466a946da5e79675845e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:52:45.493973 2390 tablet_replica.cc:439] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fda587220c88485bb5240769ffdb2d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 } } peers { permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 } } peers { permanent_uuid: "fda587220c88485bb5240769ffdb2d96" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34319 } } }
I20260926 06:52:45.494053 2390 tablet_replica.cc:442] T 31ca266582164014a09cea8afe329ba1 P 73bc817f2231454c8eb7a1545535452d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:52:45.499572 2368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.497166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:34319 (local address 127.0.191.195:35661)
0926 06:52:45.497233 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:45.497244 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:45.497284 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:45.497452 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:45.497453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:45.497454 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:52:45.497495 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.497496 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.498287 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.498287 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.498795 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:45.498796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.499230 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.499231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:45.499265 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.499276 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:52:45.499422 (+ 146us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:52:45.499424 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:52:45.499457 (+ 33us) client_negotiation.cc:240] Negotiation successful
0926 06:52:45.499487 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 06:52:45.499572 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.497231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:35661 (local address 127.0.191.195:34319)
0926 06:52:45.497266 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.497268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.497275 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:52:45.497332 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.497333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.497343 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.497449 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:52:45.497553 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.498222 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.498865 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.499161 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.499233 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.499341 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:52:45.499370 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:52:45.499417 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.499459 (+ 42us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.499477 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:52:45.501969 2036 raft_consensus.cc:1261] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e [term 1 FOLLOWER]: Refusing update from remote peer fda587220c88485bb5240769ffdb2d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:45.502197 2176 raft_consensus.cc:1261] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d [term 1 FOLLOWER]: Refusing update from remote peer fda587220c88485bb5240769ffdb2d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:45.502171 2392 consensus_queue.cc:1035] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f6eafd243c466a946da5e79675845e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 40009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:45.502439 2392 consensus_queue.cc:1035] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73bc817f2231454c8eb7a1545535452d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 40059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:45.512355 1878 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:52:45.512455 1878 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:52:45.513065 1878 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f86f3ab5ccbf4e7c99099d372cb28874: Sending DeleteTablet for 3 replicas of tablet f221cc9f618d4fbb8074da4748ef7473
I20260926 06:52:45.513289 2156 tablet_service.cc:1514] Processing DeleteTablet for tablet f221cc9f618d4fbb8074da4748ef7473 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:45 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:50236
I20260926 06:52:45.513332 2016 tablet_service.cc:1514] Processing DeleteTablet for tablet f221cc9f618d4fbb8074da4748ef7473 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:45 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:39902
I20260926 06:52:45.513394 2406 tablet_replica.cc:330] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d: stopping tablet replica
I20260926 06:52:45.513437 2406 raft_consensus.cc:2229] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:52:45.513444 2404 tablet_replica.cc:330] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e: stopping tablet replica
I20260926 06:52:45.513443 2295 tablet_service.cc:1514] Processing DeleteTablet for tablet f221cc9f618d4fbb8074da4748ef7473 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:45 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:33644
I20260926 06:52:45.513468 2406 raft_consensus.cc:2258] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:45.513490 2404 raft_consensus.cc:2229] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:52:45.513526 2404 raft_consensus.cc:2258] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:45.513535 2405 tablet_replica.cc:330] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96: stopping tablet replica
I20260926 06:52:45.513584 2405 raft_consensus.cc:2229] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:52:45.513628 2405 raft_consensus.cc:2258] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:45.513902 2404 ts_tablet_manager.cc:1905] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:45.513943 2406 ts_tablet_manager.cc:1905] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:45.515353 2404 ts_tablet_manager.cc:1918] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:52:45.515410 2404 log.cc:1196] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-0/wal/wals/f221cc9f618d4fbb8074da4748ef7473
I20260926 06:52:45.515544 2405 ts_tablet_manager.cc:1905] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:45.515626 2404 ts_tablet_manager.cc:1939] T f221cc9f618d4fbb8074da4748ef7473 P 20f6eafd243c466a946da5e79675845e: Deleting consensus metadata
I20260926 06:52:45.515913 1864 catalog_manager.cc:4841] TS 20f6eafd243c466a946da5e79675845e (127.0.191.193:40009): tablet f221cc9f618d4fbb8074da4748ef7473 (table test-table [id=09ae2f954d2e4ca09e5f256f976d7b18]) successfully deleted
I20260926 06:52:45.516925 2405 ts_tablet_manager.cc:1918] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:52:45.516984 2405 log.cc:1196] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/f221cc9f618d4fbb8074da4748ef7473
I20260926 06:52:45.517182 2405 ts_tablet_manager.cc:1939] T f221cc9f618d4fbb8074da4748ef7473 P fda587220c88485bb5240769ffdb2d96: Deleting consensus metadata
I20260926 06:52:45.517365 2406 ts_tablet_manager.cc:1918] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:52:45.517431 2406 log.cc:1196] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/f221cc9f618d4fbb8074da4748ef7473
I20260926 06:52:45.517637 2406 ts_tablet_manager.cc:1939] T f221cc9f618d4fbb8074da4748ef7473 P 73bc817f2231454c8eb7a1545535452d: Deleting consensus metadata
I20260926 06:52:45.517917 1865 catalog_manager.cc:4841] TS 73bc817f2231454c8eb7a1545535452d (127.0.191.194:40059): tablet f221cc9f618d4fbb8074da4748ef7473 (table test-table [id=09ae2f954d2e4ca09e5f256f976d7b18]) successfully deleted
I20260926 06:52:45.518204 1864 catalog_manager.cc:4841] TS fda587220c88485bb5240769ffdb2d96 (127.0.191.195:34319): tablet f221cc9f618d4fbb8074da4748ef7473 (table test-table [id=09ae2f954d2e4ca09e5f256f976d7b18]) successfully deleted
Sep 26 06:52:45 dist-test-slave-ws74 krb5kdc[1834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405565, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.191.193@KRBTEST.COM
I20260926 06:52:45.524489 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.515696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39906 (local address 127.0.191.193:40009)
0926 06:52:45.515737 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.515739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.516166 (+ 427us) server_negotiation.cc:407] Connection header received
0926 06:52:45.516223 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.516224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.516234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.516278 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:45.516821 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.517505 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.518480 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.518689 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.518785 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.521640 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:45.521647 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:45.521648 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:45.521654 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:45.523326 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:45.523690 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:45.523694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:45.523694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:45.523752 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:45.523963 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:45.523964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:45.523964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:45.524099 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:45.524263 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.524306 (+ 43us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.524383 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 06:52:45.524735 2076 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39906
I20260926 06:52:45.531999 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:45.525606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59478 (local address 127.0.191.254:44667)
0926 06:52:45.525663 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:45.525665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:45.525783 (+ 118us) server_negotiation.cc:407] Connection header received
0926 06:52:45.525809 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:45.525810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:45.525820 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:45.525883 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:45.526319 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.527097 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.527780 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:45.528069 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:45.528129 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:45.529140 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:45.529142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:45.529142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:45.529149 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:45.530601 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:45.530894 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:45.530895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:45.530895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:45.530954 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:45.531515 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:45.531516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:45.531516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:45.531644 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:45.531750 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:45.531839 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 06:52:45.531912 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260926 06:52:45.532275 1878 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59478
I20260926 06:52:45.532887 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 1951
I20260926 06:52:45.539897 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 2091
I20260926 06:52:45.545053 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 2231
I20260926 06:52:45.549355 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 1847
2026-09-26T06:52:45Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2251 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:52:45 dist-test-slave-ws74 krb5kdc[2441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:52:45 dist-test-slave-ws74 krb5kdc[2441](info): set up 2 sockets
Sep 26 06:52:45 dist-test-slave-ws74 krb5kdc[2441](info): commencing operation
krb5kdc: starting...
W20260926 06:52:46.588980 767 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:52:46 dist-test-slave-ws74 krb5kdc[2441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:52:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:52:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:52:46.700093 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:46729
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:46729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:46.779250 2454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:46.779489 2454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:46.779533 2454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:46.780669 2454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:52:46.780737 2454 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 06:52:46.780758 2454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:46.780781 2454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:46.780810 2454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:52:46.780833 2454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:52:46.782279 2454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:46729
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:46729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:46.782539 2454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:46.782755 2454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:46.787772 2462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:46.787837 2454 server_base.cc:1034] running on GCE node
W20260926 06:52:46.787786 2459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:46.787786 2460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:46.788214 2454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:46.788506 2454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:46.789669 2454 hybrid_clock.cc:648] HybridClock initialized: now 1790405566789654 us; error 45 us; skew 500 ppm
Sep 26 06:52:46 dist-test-slave-ws74 krb5kdc[2441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405566, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:46.792183 2454 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:52:46.792626 2454 webserver.cc:460] Webserver started at http://127.0.191.254:41295/ using document root <none> and password file <none>
I20260926 06:52:46.792817 2454 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:46.792857 2454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:46.792943 2454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:46.793978 2454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "e7a113f8ae3e49199f64bbe6587b475e"
format_stamp: "Formatted at 2026-09-26 06:52:46 on dist-test-slave-ws74"
server_key: "4753c0c8356492f87851af81213dabd7"
server_key_iv: "37b82071fc244848d7cb0341f6125a96"
server_key_version: "encryptionkey@0"
I20260926 06:52:46.794281 2454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "e7a113f8ae3e49199f64bbe6587b475e"
format_stamp: "Formatted at 2026-09-26 06:52:46 on dist-test-slave-ws74"
server_key: "4753c0c8356492f87851af81213dabd7"
server_key_iv: "37b82071fc244848d7cb0341f6125a96"
server_key_version: "encryptionkey@0"
I20260926 06:52:46.795521 2454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:52:46.796168 2469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:46.796350 2454 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:52:46.796422 2454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "e7a113f8ae3e49199f64bbe6587b475e"
format_stamp: "Formatted at 2026-09-26 06:52:46 on dist-test-slave-ws74"
server_key: "4753c0c8356492f87851af81213dabd7"
server_key_iv: "37b82071fc244848d7cb0341f6125a96"
server_key_version: "encryptionkey@0"
I20260926 06:52:46.796471 2454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:46.813871 2454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:46.816041 2454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:46.816145 2454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:46.820741 2454 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:46729
I20260926 06:52:46.820823 2531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:46729 every 8 connection(s)
I20260926 06:52:46.821192 2454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:52:46.821904 2532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:46.824102 2532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e: Bootstrap starting.
I20260926 06:52:46.824210 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 2454
I20260926 06:52:46.824309 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:52:46.824466 767 external_mini_cluster.cc:1428] Setting key 6d79eae21f4eb8d2527b85ab0b1781fd
I20260926 06:52:46.824838 2532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:46.825208 2532 log.cc:824] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:46.825946 2532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e: No bootstrap required, opened a new log
I20260926 06:52:46.827709 2532 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7a113f8ae3e49199f64bbe6587b475e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 46729 } }
I20260926 06:52:46.827872 2532 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:46.827903 2532 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7a113f8ae3e49199f64bbe6587b475e, State: Initialized, Role: FOLLOWER
I20260926 06:52:46.828011 2532 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7a113f8ae3e49199f64bbe6587b475e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 46729 } }
I20260926 06:52:46.828075 2532 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:52:46.828100 2532 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:52:46.828151 2532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:46.828719 2532 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7a113f8ae3e49199f64bbe6587b475e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 46729 } }
I20260926 06:52:46.828804 2532 leader_election.cc:302] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7a113f8ae3e49199f64bbe6587b475e; no voters:
I20260926 06:52:46.828979 2532 leader_election.cc:288] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:52:46.829010 2537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:46.829139 2537 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [term 1 LEADER]: Becoming Leader. State: Replica: e7a113f8ae3e49199f64bbe6587b475e, State: Running, Role: LEADER
I20260926 06:52:46.829219 2537 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7a113f8ae3e49199f64bbe6587b475e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 46729 } }
I20260926 06:52:46.829222 2532 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:52:46.829556 2538 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7a113f8ae3e49199f64bbe6587b475e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7a113f8ae3e49199f64bbe6587b475e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 46729 } } }
I20260926 06:52:46.829561 2539 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7a113f8ae3e49199f64bbe6587b475e. Latest consensus state: current_term: 1 leader_uuid: "e7a113f8ae3e49199f64bbe6587b475e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7a113f8ae3e49199f64bbe6587b475e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 46729 } } }
I20260926 06:52:46.829658 2538 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:46.829664 2539 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:46.829854 2543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:52:46.830278 2543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:52:46.831746 2543 catalog_manager.cc:1347] Generated new cluster ID: 04436850f4114aa8a41dfdfb891dd4bf
I20260926 06:52:46.831799 2543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:52:46 dist-test-slave-ws74 krb5kdc[2441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:46.835978 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:46.824923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50826 (local address 127.0.191.254:46729)
0926 06:52:46.825131 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:46.825142 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:46.825178 (+ 36us) server_negotiation.cc:407] Connection header received
0926 06:52:46.825749 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:46.825757 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:46.825816 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:46.825960 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:46.826544 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:46.828299 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:46.829161 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:46.829805 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:46.829917 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:46.832574 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:46.832578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:46.832578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:46.832589 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:46.834195 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:46.834572 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:46.834575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:46.834575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:46.834659 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:46.834860 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:46.834861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:46.834861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:46.835131 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:46.835313 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:46.835558 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 06:52:46.835731 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":86,"threads_started":1}
I20260926 06:52:46.840415 2543 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:52:46.840919 2543 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:52:46.846009 2543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e: Generated new TSK 0
I20260926 06:52:46.846172 2543 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:52:46.875789 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:46729
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:46.979277 2558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:46.979502 2558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:46.979540 2558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:46.980655 2558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:46.980720 2558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:46.980754 2558 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 06:52:46.980783 2558 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:52:46.980813 2558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:52:46.982286 2558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.191.254:46729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:46.982549 2558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:46.982764 2558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:46.987660 2563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:46.987660 2564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:46.987830 2558 server_base.cc:1034] running on GCE node
W20260926 06:52:46.987718 2566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:46.988111 2558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:46.988337 2558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:46.989509 2558 hybrid_clock.cc:648] HybridClock initialized: now 1790405566989494 us; error 31 us; skew 500 ppm
Sep 26 06:52:46 dist-test-slave-ws74 krb5kdc[2441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405566, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:46.997911 2558 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:52:46.998416 2558 webserver.cc:460] Webserver started at http://127.0.191.193:40715/ using document root <none> and password file <none>
I20260926 06:52:46.998661 2558 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:46.998729 2558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:46.998857 2558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:46.999910 2558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "777f1c96cfae4fcaae963fdc17f98fda"
format_stamp: "Formatted at 2026-09-26 06:52:46 on dist-test-slave-ws74"
server_key: "4f582f282f7622c2ffa13f3df534a4df"
server_key_iv: "01b1a22011990d7fef734eae673eae5b"
server_key_version: "encryptionkey@0"
I20260926 06:52:47.000316 2558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "777f1c96cfae4fcaae963fdc17f98fda"
format_stamp: "Formatted at 2026-09-26 06:52:46 on dist-test-slave-ws74"
server_key: "4f582f282f7622c2ffa13f3df534a4df"
server_key_iv: "01b1a22011990d7fef734eae673eae5b"
server_key_version: "encryptionkey@0"
I20260926 06:52:47.001744 2558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:52:47.002527 2573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:47.002719 2558 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:52:47.002816 2558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "777f1c96cfae4fcaae963fdc17f98fda"
format_stamp: "Formatted at 2026-09-26 06:52:46 on dist-test-slave-ws74"
server_key: "4f582f282f7622c2ffa13f3df534a4df"
server_key_iv: "01b1a22011990d7fef734eae673eae5b"
server_key_version: "encryptionkey@0"
I20260926 06:52:47.002895 2558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:47.010605 2558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:47.012357 2558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:47.012511 2558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:47.013015 2558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:47.013072 2558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:47.013110 2558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:47.013137 2558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 06:52:47 dist-test-slave-ws74 krb5kdc[2441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405566, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:47.020367 2558 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:39009
I20260926 06:52:47.020545 2688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:39009 every 8 connection(s)
I20260926 06:52:47.020746 2558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:52:47.021366 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 2558
I20260926 06:52:47.021482 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:52:47.021687 767 external_mini_cluster.cc:1428] Setting key 65720502055c08e8d58b1517df1e8ef5
I20260926 06:52:47.022739 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.013430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:46243 (local address 127.0.191.254:46729)
0926 06:52:47.013492 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.013495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.013683 (+ 188us) server_negotiation.cc:407] Connection header received
0926 06:52:47.014181 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.014183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.014195 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.014245 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:47.014837 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.015348 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.016367 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.016577 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.016654 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.019411 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:47.019413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:47.019413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:47.019421 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:47.021144 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.021639 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.021640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.021640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.021696 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.022017 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.022018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.022018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.022148 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:47.022212 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.022587 (+ 375us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.022666 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 06:52:47.022981 2582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.013418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:46729 (local address 127.0.191.193:46243)
0926 06:52:47.013573 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.013605 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.014072 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.014315 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.014318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.014559 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:47.014729 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.014733 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.015450 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.015451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.016260 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.016263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.016672 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.016673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.016836 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.017344 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:47.017352 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:47.019311 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:47.021222 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.021225 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.021227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.021568 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.021803 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.021804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.021805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.021937 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.022268 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:47.022275 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:47.022429 (+ 154us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.022540 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.022712 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260926 06:52:47.023969 2689 heartbeater.cc:347] Connected to a master server at 127.0.191.254:46729
I20260926 06:52:47.024051 2689 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:47.024225 2689 heartbeater.cc:511] Master 127.0.191.254:46729 requested a full tablet report, sending...
I20260926 06:52:47.024748 2485 ts_manager.cc:194] Registered new tserver with Master: 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009)
I20260926 06:52:47.025703 2485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:46243
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
I20260926 06:52:47.030920 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.025007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:45009 (local address 127.0.191.254:46729)
0926 06:52:47.025045 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.025047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.025153 (+ 106us) server_negotiation.cc:407] Connection header received
0926 06:52:47.025223 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.025224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.025233 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.025281 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:47.025658 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.026295 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.027040 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.027255 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.027333 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.028230 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:47.028231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:47.028232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:47.028238 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:47.029924 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.030293 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.030295 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.030295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.030345 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.030547 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.030548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.030548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.030649 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:47.030717 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.030808 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.030863 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:52:47.030931 2696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.024960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:46729 (local address 127.0.191.193:45009)
0926 06:52:47.025084 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.025093 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.025166 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.025283 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.025284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.025514 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:47.025582 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.025584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.026404 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.026404 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.026957 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.026958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.027330 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.027331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.027402 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.027841 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:47.027846 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:47.028127 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:47.029997 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.029998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.029998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.030215 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.030419 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.030420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.030420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.030483 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.030701 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:47.030702 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:47.030744 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.030791 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.030862 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:52:47.045943 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:46729
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:47.138855 2698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:47.139125 2698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:47.139168 2698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:47.141044 2698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:47.141119 2698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:47.141155 2698 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 06:52:47.141180 2698 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:52:47.141207 2698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:52:47.142899 2698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.191.254:46729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:47.143181 2698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:47.143415 2698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:47.148964 2704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:47.148974 2703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:47.149091 2698 server_base.cc:1034] running on GCE node
W20260926 06:52:47.149247 2706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:47.149601 2698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:47.149853 2698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:47.151036 2698 hybrid_clock.cc:648] HybridClock initialized: now 1790405567151023 us; error 31 us; skew 500 ppm
Sep 26 06:52:47 dist-test-slave-ws74 krb5kdc[2441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405567, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:47.153596 2698 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:52:47.154107 2698 webserver.cc:460] Webserver started at http://127.0.191.194:39071/ using document root <none> and password file <none>
I20260926 06:52:47.154337 2698 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:47.154407 2698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:47.154536 2698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:47.155628 2698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "478e14a4b1114d83ab39fcc73f078f3e"
format_stamp: "Formatted at 2026-09-26 06:52:47 on dist-test-slave-ws74"
server_key: "74ad5e18939e9e6f3d81ae841a712817"
server_key_iv: "1d2d9e9edd5d702d7f007106effd97bd"
server_key_version: "encryptionkey@0"
I20260926 06:52:47.155992 2698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "478e14a4b1114d83ab39fcc73f078f3e"
format_stamp: "Formatted at 2026-09-26 06:52:47 on dist-test-slave-ws74"
server_key: "74ad5e18939e9e6f3d81ae841a712817"
server_key_iv: "1d2d9e9edd5d702d7f007106effd97bd"
server_key_version: "encryptionkey@0"
I20260926 06:52:47.157337 2698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:52:47.158102 2713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:47.158262 2698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:52:47.158375 2698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "478e14a4b1114d83ab39fcc73f078f3e"
format_stamp: "Formatted at 2026-09-26 06:52:47 on dist-test-slave-ws74"
server_key: "74ad5e18939e9e6f3d81ae841a712817"
server_key_iv: "1d2d9e9edd5d702d7f007106effd97bd"
server_key_version: "encryptionkey@0"
I20260926 06:52:47.158461 2698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:47.170600 2698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:47.172250 2698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:47.172396 2698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:47.172865 2698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:47.172932 2698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:47.172991 2698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:47.173020 2698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:47.179659 2698 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:43509
I20260926 06:52:47.179752 2828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:43509 every 8 connection(s)
Sep 26 06:52:47 dist-test-slave-ws74 krb5kdc[2441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405567, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:47.180034 2698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:52:47.182686 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.173357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:59447 (local address 127.0.191.254:46729)
0926 06:52:47.173436 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.173438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.173625 (+ 187us) server_negotiation.cc:407] Connection header received
0926 06:52:47.174104 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.174105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.174117 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.174169 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:47.174922 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.175723 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.177356 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.177736 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.177779 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.180470 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:47.180471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:47.180472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:47.180477 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:47.181652 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.181990 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.181991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.181991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.182037 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.182238 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.182238 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.182239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.182328 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:47.182377 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.182571 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.182626 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 06:52:47.182924 2722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.173345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:46729 (local address 127.0.191.194:59447)
0926 06:52:47.173519 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.173550 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.174003 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.174254 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.174262 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.174566 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:47.174840 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.174845 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.175823 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.175824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.177244 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.177247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.177861 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.177862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.177996 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.178521 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:47.178536 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:47.180385 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:47.181717 (+ 1332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.181722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.181724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.181926 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.182093 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.182093 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.182094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.182185 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.182381 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:47.182384 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:47.182481 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.182561 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.182683 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":109,"threads_started":1}
I20260926 06:52:47.183399 2829 heartbeater.cc:347] Connected to a master server at 127.0.191.254:46729
I20260926 06:52:47.183482 2829 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:47.183660 2829 heartbeater.cc:511] Master 127.0.191.254:46729 requested a full tablet report, sending...
I20260926 06:52:47.184020 2486 ts_manager.cc:194] Registered new tserver with Master: 478e14a4b1114d83ab39fcc73f078f3e (127.0.191.194:43509)
I20260926 06:52:47.184593 2486 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:59447
I20260926 06:52:47.185806 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 2698
I20260926 06:52:47.185917 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:52:47.186098 767 external_mini_cluster.cc:1428] Setting key 5e877432b9b4b44517ab84ae305b023d
I20260926 06:52:47.191207 2835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.184539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:46729 (local address 127.0.191.194:53451)
0926 06:52:47.184703 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.184721 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.184790 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.184963 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.184963 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.185115 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:47.185166 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.185167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.186167 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.186168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.186829 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.186831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.187326 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.187326 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.187377 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.187795 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:47.187800 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:47.188227 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:47.190013 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.190014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.190014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.190260 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.190456 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.190457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.190457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.190525 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.190825 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:47.190826 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:47.190898 (+ 72us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.190992 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.191096 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
I20260926 06:52:47.191233 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.184626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:53451 (local address 127.0.191.254:46729)
0926 06:52:47.184686 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.184689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.184766 (+ 77us) server_negotiation.cc:407] Connection header received
0926 06:52:47.184847 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.184847 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.184856 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.184945 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:47.185242 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.186062 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.186900 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.187244 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.187331 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.188313 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:47.188314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:47.188314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:47.188320 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:47.189883 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.190335 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.190335 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.190336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.190386 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.190591 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.190592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.190592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.190721 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:47.190842 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.191067 (+ 225us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.191142 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:52:47.212438 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:46729
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:47.291246 2838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:47.291429 2838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:47.291452 2838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:47.292574 2838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:47.292613 2838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:47.292634 2838 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 06:52:47.292681 2838 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:52:47.292709 2838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:52:47.294142 2838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.191.254:46729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:47.294409 2838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:47.294627 2838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:47.299510 2844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:47.299520 2843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:47.299525 2846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:47.299664 2838 server_base.cc:1034] running on GCE node
I20260926 06:52:47.299847 2838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:47.300092 2838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:47.301256 2838 hybrid_clock.cc:648] HybridClock initialized: now 1790405567301245 us; error 28 us; skew 500 ppm
Sep 26 06:52:47 dist-test-slave-ws74 krb5kdc[2441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405567, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:47.303584 2838 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:52:47.304014 2838 webserver.cc:460] Webserver started at http://127.0.191.195:37807/ using document root <none> and password file <none>
I20260926 06:52:47.304245 2838 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:47.304287 2838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:47.304425 2838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:47.305469 2838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "a0f5854981be45b891c450c46830c20a"
format_stamp: "Formatted at 2026-09-26 06:52:47 on dist-test-slave-ws74"
server_key: "b0f0d4b47197b5a354f3e85dfced7b01"
server_key_iv: "bc3c28c0298070ad36ce95f76bce79c3"
server_key_version: "encryptionkey@0"
I20260926 06:52:47.305864 2838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "a0f5854981be45b891c450c46830c20a"
format_stamp: "Formatted at 2026-09-26 06:52:47 on dist-test-slave-ws74"
server_key: "b0f0d4b47197b5a354f3e85dfced7b01"
server_key_iv: "bc3c28c0298070ad36ce95f76bce79c3"
server_key_version: "encryptionkey@0"
I20260926 06:52:47.307183 2838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:52:47.307861 2853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:47.308038 2838 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:52:47.308105 2838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "a0f5854981be45b891c450c46830c20a"
format_stamp: "Formatted at 2026-09-26 06:52:47 on dist-test-slave-ws74"
server_key: "b0f0d4b47197b5a354f3e85dfced7b01"
server_key_iv: "bc3c28c0298070ad36ce95f76bce79c3"
server_key_version: "encryptionkey@0"
I20260926 06:52:47.308192 2838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:47.316488 2838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:47.318269 2838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:47.318411 2838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:47.318948 2838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:47.319013 2838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:47.319057 2838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:47.319085 2838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 06:52:47 dist-test-slave-ws74 krb5kdc[2441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405567, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:47.326181 2838 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:42499
I20260926 06:52:47.326251 2968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:42499 every 8 connection(s)
I20260926 06:52:47.326581 2838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:52:47.328240 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.319582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:58891 (local address 127.0.191.254:46729)
0926 06:52:47.319660 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.319663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.319822 (+ 159us) server_negotiation.cc:407] Connection header received
0926 06:52:47.320249 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.320251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.320265 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.320344 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:47.321159 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.322041 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.323038 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.323291 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.323380 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.325591 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:47.325593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:47.325593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:47.325601 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:47.327189 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.327525 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.327525 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.327526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.327574 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.327769 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.327769 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.327770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.327860 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:47.327909 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.328116 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.328172 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 06:52:47.328406 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.319529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:46729 (local address 127.0.191.195:58891)
0926 06:52:47.319697 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.319733 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.320157 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.320437 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.320441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.320796 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:47.321057 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.321063 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.322134 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.322135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.322917 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.322919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.323400 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.323401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.323523 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.323972 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:47.323998 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:47.325517 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:47.327260 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.327264 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.327265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.327460 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.327629 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.327629 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.327630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.327716 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.327912 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:47.327916 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:47.328016 (+ 100us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.328091 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.328203 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260926 06:52:47.328862 2969 heartbeater.cc:347] Connected to a master server at 127.0.191.254:46729
I20260926 06:52:47.328961 2969 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:47.329121 2969 heartbeater.cc:511] Master 127.0.191.254:46729 requested a full tablet report, sending...
I20260926 06:52:47.329458 2486 ts_manager.cc:194] Registered new tserver with Master: a0f5854981be45b891c450c46830c20a (127.0.191.195:42499)
I20260926 06:52:47.330011 2486 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:58891
I20260926 06:52:47.335001 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.329987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:48207 (local address 127.0.191.254:46729)
0926 06:52:47.330029 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.330031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.330142 (+ 111us) server_negotiation.cc:407] Connection header received
0926 06:52:47.330216 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.330217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.330225 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.330266 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:47.330590 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.331099 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.331795 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.331995 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.332067 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.332822 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:47.332823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:47.332824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:47.332830 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:47.333930 (+ 1100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.334273 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.334274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.334274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.334339 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.334526 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.334527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.334527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.334626 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:47.334690 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.334855 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.334920 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:52:47.335016 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.329930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:46729 (local address 127.0.191.195:48207)
0926 06:52:47.330084 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.330095 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.330155 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.330281 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.330282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.330441 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:47.330516 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.330517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.331188 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.331189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.331694 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.331695 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.332066 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.332067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.332143 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.332462 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:47.332466 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:47.332738 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:47.334022 (+ 1284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.334023 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.334023 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.334185 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.334402 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.334403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.334403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.334456 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.334694 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:47.334695 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:47.334740 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.334830 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.334922 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":103,"threads_started":1}
I20260926 06:52:47.336550 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 2838
I20260926 06:52:47.336638 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:52:47.336804 767 external_mini_cluster.cc:1428] Setting key 9adafe9e5bbd9f897ed9c277d6c7512b
I20260926 06:52:47.337567 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 06:52:47 dist-test-slave-ws74 krb5kdc[2441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 06:52:47 dist-test-slave-ws74 krb5kdc[2441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:47.358770 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.352321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50840 (local address 127.0.191.254:46729)
0926 06:52:47.352391 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.352393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.352406 (+ 13us) server_negotiation.cc:407] Connection header received
0926 06:52:47.352494 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.352495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.352505 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.352554 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:47.353103 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.353652 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.354381 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.354563 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.354634 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.356722 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:47.356723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:47.356724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:47.356730 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:47.357906 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.358202 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.358203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.358203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.358248 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.358402 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.358402 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.358402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.358490 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:47.358564 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.358641 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.358704 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 06:52:47.360237 2486 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:52:47.361992 2486 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:52:47.368275 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.364178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:42499 (local address 127.0.0.1:36160)
0926 06:52:47.364525 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.364534 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.364586 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.364770 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.364771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.364772 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:47.364820 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.364821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.366105 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.366105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.367285 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.367287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.367961 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.367962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.368037 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.368049 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.368147 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.368198 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":45,"thread_start_us":64,"threads_started":1}
I20260926 06:52:47.368296 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.364350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36160 (local address 127.0.191.195:42499)
0926 06:52:47.364485 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.364490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.364592 (+ 102us) server_negotiation.cc:407] Connection header received
0926 06:52:47.364651 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.364654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.364668 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.364894 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:47.364951 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.366014 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.367369 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.367860 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.367957 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.368018 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.368165 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.368203 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
I20260926 06:52:47.368525 2990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.364363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39009 (local address 127.0.0.1:45572)
0926 06:52:47.364729 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.364738 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.364787 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.365034 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.365035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.365037 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:47.365082 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.365083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.365983 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.365984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.367327 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.367329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.368290 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.368291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.368352 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.368356 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.368411 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.368460 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":6,"thread_start_us":59,"threads_started":1}
I20260926 06:52:47.368824 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.364562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45572 (local address 127.0.191.193:39009)
0926 06:52:47.364697 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.364702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.364843 (+ 141us) server_negotiation.cc:407] Connection header received
0926 06:52:47.364885 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.364887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.364904 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.364974 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:47.365164 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.365896 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.367424 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.368143 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.368206 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.368261 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.368695 (+ 434us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.368734 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260926 06:52:47.368957 2903 tablet_service.cc:1467] Processing CreateTablet for tablet ec41b1d940014456a5a6297cc8493693 (DEFAULT_TABLE table=test-table [id=dc9654a21857462c931bfbe5fdaa14fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:47.369302 2903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec41b1d940014456a5a6297cc8493693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:47.369423 2623 tablet_service.cc:1467] Processing CreateTablet for tablet ec41b1d940014456a5a6297cc8493693 (DEFAULT_TABLE table=test-table [id=dc9654a21857462c931bfbe5fdaa14fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:47.369642 2987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.364170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:43509 (local address 127.0.0.1:32828)
0926 06:52:47.364330 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.364366 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.364427 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.364750 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.364753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.364755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:47.364814 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.364816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.367085 (+ 2269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.367086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.368122 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.368125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.369415 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.369416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.369476 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.369479 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.369536 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.369579 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":57,"threads_started":1}
I20260926 06:52:47.369765 2623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec41b1d940014456a5a6297cc8493693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:47.369921 2988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.364268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32828 (local address 127.0.191.194:43509)
0926 06:52:47.364516 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.364519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.364528 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:52:47.364567 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.364569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.364587 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.364673 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:47.364898 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.366008 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.368231 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.369241 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.369310 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.369361 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.369566 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.369830 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":86,"threads_started":1}
I20260926 06:52:47.370502 2763 tablet_service.cc:1467] Processing CreateTablet for tablet ec41b1d940014456a5a6297cc8493693 (DEFAULT_TABLE table=test-table [id=dc9654a21857462c931bfbe5fdaa14fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:47.370820 2763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec41b1d940014456a5a6297cc8493693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:47.372232 2992 tablet_bootstrap.cc:492] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a: Bootstrap starting.
I20260926 06:52:47.373031 2992 tablet_bootstrap.cc:654] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:47.373234 2993 tablet_bootstrap.cc:492] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda: Bootstrap starting.
I20260926 06:52:47.373425 2992 log.cc:824] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:47.373864 2993 tablet_bootstrap.cc:654] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:47.374133 2993 log.cc:824] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:47.374212 2992 tablet_bootstrap.cc:492] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a: No bootstrap required, opened a new log
I20260926 06:52:47.374290 2992 ts_tablet_manager.cc:1397] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:52:47.374550 2995 tablet_bootstrap.cc:492] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e: Bootstrap starting.
I20260926 06:52:47.374821 2993 tablet_bootstrap.cc:492] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda: No bootstrap required, opened a new log
I20260926 06:52:47.374897 2993 ts_tablet_manager.cc:1397] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:52:47.375141 2995 tablet_bootstrap.cc:654] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:47.375474 2995 log.cc:824] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:47.376230 2995 tablet_bootstrap.cc:492] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e: No bootstrap required, opened a new log
I20260926 06:52:47.376302 2995 ts_tablet_manager.cc:1397] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:52:47.376395 2992 raft_consensus.cc:348] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } }
I20260926 06:52:47.376557 2992 raft_consensus.cc:374] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:47.376601 2992 raft_consensus.cc:729] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f5854981be45b891c450c46830c20a, State: Initialized, Role: FOLLOWER
I20260926 06:52:47.376725 2992 consensus_queue.cc:260] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } }
I20260926 06:52:47.376708 2993 raft_consensus.cc:348] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } }
I20260926 06:52:47.376889 2993 raft_consensus.cc:374] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:47.376932 2993 raft_consensus.cc:729] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 777f1c96cfae4fcaae963fdc17f98fda, State: Initialized, Role: FOLLOWER
I20260926 06:52:47.376989 2992 ts_tablet_manager.cc:1428] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:52:47.377048 2993 consensus_queue.cc:260] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } }
I20260926 06:52:47.377122 2969 heartbeater.cc:503] Master 127.0.191.254:46729 was elected leader, sending a full tablet report...
I20260926 06:52:47.377307 2993 ts_tablet_manager.cc:1428] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 06:52:47.377390 2689 heartbeater.cc:503] Master 127.0.191.254:46729 was elected leader, sending a full tablet report...
I20260926 06:52:47.382364 2995 raft_consensus.cc:348] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } }
I20260926 06:52:47.382526 2995 raft_consensus.cc:374] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:47.382572 2995 raft_consensus.cc:729] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478e14a4b1114d83ab39fcc73f078f3e, State: Initialized, Role: FOLLOWER
I20260926 06:52:47.382691 2995 consensus_queue.cc:260] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } }
I20260926 06:52:47.382961 2995 ts_tablet_manager.cc:1428] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260926 06:52:47.383085 2829 heartbeater.cc:503] Master 127.0.191.254:46729 was elected leader, sending a full tablet report...
W20260926 06:52:47.430563 2830 tablet.cc:2369] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:52:47.432987 3000 raft_consensus.cc:482] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:52:47.433135 3000 raft_consensus.cc:504] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } }
I20260926 06:52:47.433564 3000 leader_election.cc:288] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0f5854981be45b891c450c46830c20a (127.0.191.195:42499), 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009)
I20260926 06:52:47.436375 2722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.433766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39009 (local address 127.0.191.194:53583)
0926 06:52:47.433820 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.433831 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.433875 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.434048 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.434049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.434051 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:47.434121 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.434123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.434789 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.434790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.435531 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.435533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.436086 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.436087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.436173 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.436176 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.436252 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.436292 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 06:52:47.436497 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.433873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:53583 (local address 127.0.191.193:39009)
0926 06:52:47.433922 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.433924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.433932 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:52:47.433966 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.433967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.433974 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.434041 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:47.434202 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.434692 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.435622 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.436000 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.436088 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.436096 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.436355 (+ 259us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.436424 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 06:52:47.436547 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.433789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:42499 (local address 127.0.191.194:51383)
0926 06:52:47.433971 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.433990 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.434085 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.434243 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.434244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.434245 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:47.434320 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.434321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.434927 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.434928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.435681 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.435683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.436277 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.436278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.436358 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.436361 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.436422 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.436461 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":105,"threads_started":1}
I20260926 06:52:47.436543 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.433880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:51383 (local address 127.0.191.195:42499)
0926 06:52:47.433932 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.433935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.434068 (+ 133us) server_negotiation.cc:407] Connection header received
0926 06:52:47.434151 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.434152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.434168 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.434239 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:47.434392 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.434844 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.435775 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.436189 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.436277 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.436285 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.436440 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.436474 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 06:52:47.436800 2643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec41b1d940014456a5a6297cc8493693" candidate_uuid: "478e14a4b1114d83ab39fcc73f078f3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777f1c96cfae4fcaae963fdc17f98fda" is_pre_election: true
I20260926 06:52:47.436923 2643 raft_consensus.cc:2454] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478e14a4b1114d83ab39fcc73f078f3e in term 0.
I20260926 06:52:47.437196 2923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec41b1d940014456a5a6297cc8493693" candidate_uuid: "478e14a4b1114d83ab39fcc73f078f3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0f5854981be45b891c450c46830c20a" is_pre_election: true
I20260926 06:52:47.437222 2714 leader_election.cc:302] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478e14a4b1114d83ab39fcc73f078f3e, 777f1c96cfae4fcaae963fdc17f98fda; no voters:
I20260926 06:52:47.437350 2923 raft_consensus.cc:2454] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478e14a4b1114d83ab39fcc73f078f3e in term 0.
I20260926 06:52:47.437397 3000 raft_consensus.cc:2781] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:52:47.437464 3000 raft_consensus.cc:482] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:52:47.437525 3000 raft_consensus.cc:3037] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:47.494720 3000 raft_consensus.cc:504] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } }
I20260926 06:52:47.495173 3000 leader_election.cc:288] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [CANDIDATE]: Term 1 election: Requested vote from peers a0f5854981be45b891c450c46830c20a (127.0.191.195:42499), 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009)
I20260926 06:52:47.495210 2923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec41b1d940014456a5a6297cc8493693" candidate_uuid: "478e14a4b1114d83ab39fcc73f078f3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0f5854981be45b891c450c46830c20a"
I20260926 06:52:47.495226 2643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec41b1d940014456a5a6297cc8493693" candidate_uuid: "478e14a4b1114d83ab39fcc73f078f3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777f1c96cfae4fcaae963fdc17f98fda"
I20260926 06:52:47.495332 2643 raft_consensus.cc:3037] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:47.495318 2923 raft_consensus.cc:3037] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:47.496282 2923 raft_consensus.cc:2454] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478e14a4b1114d83ab39fcc73f078f3e in term 1.
I20260926 06:52:47.496284 2643 raft_consensus.cc:2454] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478e14a4b1114d83ab39fcc73f078f3e in term 1.
I20260926 06:52:47.496464 2714 leader_election.cc:302] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478e14a4b1114d83ab39fcc73f078f3e, 777f1c96cfae4fcaae963fdc17f98fda; no voters:
I20260926 06:52:47.496587 3000 raft_consensus.cc:2781] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:47.496759 3000 raft_consensus.cc:686] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 LEADER]: Becoming Leader. State: Replica: 478e14a4b1114d83ab39fcc73f078f3e, State: Running, Role: LEADER
I20260926 06:52:47.496865 3000 consensus_queue.cc:237] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } }
I20260926 06:52:47.497838 2486 catalog_manager.cc:5495] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 478e14a4b1114d83ab39fcc73f078f3e (127.0.191.194). New cstate: current_term: 1 leader_uuid: "478e14a4b1114d83ab39fcc73f078f3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } health_report { overall_health: UNKNOWN } } }
W20260926 06:52:47.521602 2690 tablet.cc:2369] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 06:52:47 dist-test-slave-ws74 krb5kdc[2441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405566, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:47.536885 3009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.530051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:46729 (local address 127.0.0.1:50852)
0926 06:52:47.530214 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.530224 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.530287 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.530405 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.530405 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.530774 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:47.530820 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.530822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.531452 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.531453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.531965 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.531966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.532277 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.532278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.532316 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.532897 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:47.532902 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:47.534260 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:47.535820 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.535827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.535829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.536031 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.536224 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:47.536225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:47.536225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:47.536283 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:47.536568 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:47.536575 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:47.536640 (+ 65us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.536719 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.536811 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260926 06:52:47.537029 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.530036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.0.191.254:46729)
0926 06:52:47.530089 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.530091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.530300 (+ 209us) server_negotiation.cc:407] Connection header received
0926 06:52:47.530328 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.530329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.530338 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.530390 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:47.530910 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.531384 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.532038 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.532212 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.532264 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.534364 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:47.534365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:47.534366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:47.534372 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:47.535722 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.536104 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.536105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.536105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.536159 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:47.536351 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:47.536351 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:47.536352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:47.536488 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:47.536569 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.536747 (+ 178us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.536892 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":27}
I20260926 06:52:47.539206 2486 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:50852:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 06:52:47.539433 2486 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:52:47.541003 2623 tablet_service.cc:1467] Processing CreateTablet for tablet b2e68f788d1640a6aa5d71c7120a8576 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9480a7068924b589436b5f1821b4474]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:52:47.541028 2903 tablet_service.cc:1467] Processing CreateTablet for tablet b2e68f788d1640a6aa5d71c7120a8576 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9480a7068924b589436b5f1821b4474]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:52:47.541189 2623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2e68f788d1640a6aa5d71c7120a8576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:47.541188 2903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2e68f788d1640a6aa5d71c7120a8576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:47.541186 2763 tablet_service.cc:1467] Processing CreateTablet for tablet b2e68f788d1640a6aa5d71c7120a8576 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9480a7068924b589436b5f1821b4474]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:52:47.541334 2763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2e68f788d1640a6aa5d71c7120a8576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:47.542491 2992 tablet_bootstrap.cc:492] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: Bootstrap starting.
I20260926 06:52:47.542689 2993 tablet_bootstrap.cc:492] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda: Bootstrap starting.
I20260926 06:52:47.543089 2993 tablet_bootstrap.cc:654] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:47.543089 2992 tablet_bootstrap.cc:654] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:47.543437 2995 tablet_bootstrap.cc:492] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e: Bootstrap starting.
I20260926 06:52:47.543758 2993 tablet_bootstrap.cc:492] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda: No bootstrap required, opened a new log
I20260926 06:52:47.543800 2993 ts_tablet_manager.cc:1397] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 06:52:47.543831 2995 tablet_bootstrap.cc:654] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:47.543850 2992 tablet_bootstrap.cc:492] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: No bootstrap required, opened a new log
I20260926 06:52:47.543885 2992 ts_tablet_manager.cc:1397] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:52:47.543951 2993 raft_consensus.cc:348] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.543994 2992 raft_consensus.cc:348] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.544025 2993 raft_consensus.cc:374] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:47.544037 2992 raft_consensus.cc:374] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:47.544051 2992 raft_consensus.cc:729] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f5854981be45b891c450c46830c20a, State: Initialized, Role: FOLLOWER
I20260926 06:52:47.544047 2993 raft_consensus.cc:729] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 777f1c96cfae4fcaae963fdc17f98fda, State: Initialized, Role: FOLLOWER
I20260926 06:52:47.544100 2993 consensus_queue.cc:260] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.544112 2992 consensus_queue.cc:260] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.544222 2992 ts_tablet_manager.cc:1428] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:47.544227 2993 ts_tablet_manager.cc:1428] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 06:52:47.544267 2999 tablet_replica.cc:439] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } }
I20260926 06:52:47.544375 2999 tablet_replica.cc:442] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:52:47.544526 2998 tablet_replica.cc:439] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } }
I20260926 06:52:47.544637 2998 tablet_replica.cc:442] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:52:47.544747 2995 tablet_bootstrap.cc:492] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e: No bootstrap required, opened a new log
I20260926 06:52:47.544795 2995 ts_tablet_manager.cc:1397] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:52:47.544940 2995 raft_consensus.cc:348] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.545034 2995 raft_consensus.cc:374] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:47.545058 2995 raft_consensus.cc:729] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478e14a4b1114d83ab39fcc73f078f3e, State: Initialized, Role: FOLLOWER
I20260926 06:52:47.545104 2995 consensus_queue.cc:260] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.545190 2995 ts_tablet_manager.cc:1428] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:47.545233 3000 tablet_replica.cc:439] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } }
I20260926 06:52:47.545320 3000 tablet_replica.cc:442] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e: This TxnStatusTablet replica's current role is: FOLLOWER
W20260926 06:52:47.577167 2970 tablet.cc:2369] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:52:47.619581 2998 raft_consensus.cc:482] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:52:47.619690 2998 raft_consensus.cc:504] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.620049 2998 leader_election.cc:288] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009), 478e14a4b1114d83ab39fcc73f078f3e (127.0.191.194:43509)
I20260926 06:52:47.624011 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.620313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:35219 (local address 127.0.191.193:39009)
0926 06:52:47.620368 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.620370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.620379 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:52:47.620409 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.620410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.620419 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.620483 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:47.620649 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.621513 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.623054 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.623607 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.623682 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.623690 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.623897 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.623956 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 06:52:47.624006 2988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.620469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:39599 (local address 127.0.191.194:43509)
0926 06:52:47.620527 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.620529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.620566 (+ 37us) server_negotiation.cc:407] Connection header received
0926 06:52:47.620649 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.620650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.620658 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.620720 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:52:47.620839 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.621530 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.623050 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.623607 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.623682 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.623690 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.623897 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.623936 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:52:47.624054 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.620235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39009 (local address 127.0.191.195:35219)
0926 06:52:47.620292 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.620303 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.620348 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.620506 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.620507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.620509 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:47.620579 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.620581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.621632 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.621634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.622951 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.622954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.623700 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.623701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.623830 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.623832 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.623890 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.623939 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":67}
I20260926 06:52:47.624167 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.620349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:43509 (local address 127.0.191.195:39599)
0926 06:52:47.620511 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.620521 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.620576 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.620729 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.620729 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.620731 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:52:47.620779 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.620780 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.621632 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.621634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.622951 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.622954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.623721 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.623722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.623824 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.623829 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.623890 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.623939 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":45,"thread_start_us":98,"threads_started":1}
I20260926 06:52:47.624428 2783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e68f788d1640a6aa5d71c7120a8576" candidate_uuid: "a0f5854981be45b891c450c46830c20a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478e14a4b1114d83ab39fcc73f078f3e" is_pre_election: true
I20260926 06:52:47.624534 2783 raft_consensus.cc:2454] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f5854981be45b891c450c46830c20a in term 0.
I20260926 06:52:47.624643 2643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e68f788d1640a6aa5d71c7120a8576" candidate_uuid: "a0f5854981be45b891c450c46830c20a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777f1c96cfae4fcaae963fdc17f98fda" is_pre_election: true
I20260926 06:52:47.624737 2643 raft_consensus.cc:2454] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f5854981be45b891c450c46830c20a in term 0.
I20260926 06:52:47.624752 2854 leader_election.cc:302] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478e14a4b1114d83ab39fcc73f078f3e, a0f5854981be45b891c450c46830c20a; no voters:
I20260926 06:52:47.624894 2998 raft_consensus.cc:2781] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:52:47.624936 2998 raft_consensus.cc:482] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:52:47.624958 2998 raft_consensus.cc:3037] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:47.625706 2998 raft_consensus.cc:504] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.625833 2998 leader_election.cc:288] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [CANDIDATE]: Term 1 election: Requested vote from peers 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009), 478e14a4b1114d83ab39fcc73f078f3e (127.0.191.194:43509)
I20260926 06:52:47.626008 2783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e68f788d1640a6aa5d71c7120a8576" candidate_uuid: "a0f5854981be45b891c450c46830c20a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478e14a4b1114d83ab39fcc73f078f3e"
I20260926 06:52:47.626006 2643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e68f788d1640a6aa5d71c7120a8576" candidate_uuid: "a0f5854981be45b891c450c46830c20a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777f1c96cfae4fcaae963fdc17f98fda"
I20260926 06:52:47.626075 2643 raft_consensus.cc:3037] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:47.626093 2783 raft_consensus.cc:3037] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:47.626657 2643 raft_consensus.cc:2454] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f5854981be45b891c450c46830c20a in term 1.
I20260926 06:52:47.626756 2783 raft_consensus.cc:2454] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f5854981be45b891c450c46830c20a in term 1.
I20260926 06:52:47.626808 2854 leader_election.cc:302] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 777f1c96cfae4fcaae963fdc17f98fda, a0f5854981be45b891c450c46830c20a; no voters:
I20260926 06:52:47.626891 2998 raft_consensus.cc:2781] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:47.627038 2998 raft_consensus.cc:686] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [term 1 LEADER]: Becoming Leader. State: Replica: a0f5854981be45b891c450c46830c20a, State: Running, Role: LEADER
I20260926 06:52:47.627149 2998 consensus_queue.cc:237] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.627563 3014 tablet_replica.cc:439] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: TxnStatusTablet state changed. Reason: New leader a0f5854981be45b891c450c46830c20a. Latest consensus state: current_term: 1 leader_uuid: "a0f5854981be45b891c450c46830c20a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } }
I20260926 06:52:47.627645 3014 tablet_replica.cc:442] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:52:47.627816 3016 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 06:52:47.627939 3016 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 06:52:47.628007 2485 catalog_manager.cc:5495] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a reported cstate change: term changed from 0 to 1, leader changed from <none> to a0f5854981be45b891c450c46830c20a (127.0.191.195). New cstate: current_term: 1 leader_uuid: "a0f5854981be45b891c450c46830c20a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260926 06:52:47.637971 3009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.635264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:42499 (local address 127.0.0.1:36170)
0926 06:52:47.635337 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.635349 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.635408 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.635566 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.635566 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.635567 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:52:47.635606 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.635608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.636402 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.636402 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.637059 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.637061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.637523 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.637524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.637570 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.637583 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:52:47.637849 (+ 266us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:52:47.637854 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.637882 (+ 28us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.637916 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 06:52:47.637977 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.635336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36170 (local address 127.0.191.195:42499)
0926 06:52:47.635383 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.635385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.635395 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:52:47.635466 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.635467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.635484 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.635566 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:52:47.635645 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.636322 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.637136 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.637447 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.637523 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.637668 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:52:47.637787 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:52:47.637864 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.637887 (+ 23us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.637907 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:52:47.639637 2783 raft_consensus.cc:1261] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Refusing update from remote peer a0f5854981be45b891c450c46830c20a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:47.639925 2643 raft_consensus.cc:1261] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Refusing update from remote peer a0f5854981be45b891c450c46830c20a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:47.639940 3014 consensus_queue.cc:1035] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [LEADER]: Connected to new peer: Peer: permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:47.640240 3014 consensus_queue.cc:1035] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a [LEADER]: Connected to new peer: Peer: permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:47.641450 3000 tablet_replica.cc:439] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e: TxnStatusTablet state changed. Reason: New leader a0f5854981be45b891c450c46830c20a. Latest consensus state: current_term: 1 leader_uuid: "a0f5854981be45b891c450c46830c20a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } }
I20260926 06:52:47.641466 2999 tablet_replica.cc:439] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda: TxnStatusTablet state changed. Reason: New leader a0f5854981be45b891c450c46830c20a. Latest consensus state: current_term: 1 leader_uuid: "a0f5854981be45b891c450c46830c20a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } }
I20260926 06:52:47.641533 3000 tablet_replica.cc:442] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:52:47.641554 2999 tablet_replica.cc:442] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:52:47.642350 2998 tablet_replica.cc:439] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0f5854981be45b891c450c46830c20a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } }
I20260926 06:52:47.642436 2998 tablet_replica.cc:442] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:52:47.642599 3000 tablet_replica.cc:439] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0f5854981be45b891c450c46830c20a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } }
I20260926 06:52:47.642688 3000 tablet_replica.cc:442] T b2e68f788d1640a6aa5d71c7120a8576 P 478e14a4b1114d83ab39fcc73f078f3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:52:47.643625 3014 tablet_replica.cc:439] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0f5854981be45b891c450c46830c20a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } }
I20260926 06:52:47.643700 3014 tablet_replica.cc:442] T b2e68f788d1640a6aa5d71c7120a8576 P a0f5854981be45b891c450c46830c20a: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:52:47.644100 2999 tablet_replica.cc:439] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0f5854981be45b891c450c46830c20a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } } }
I20260926 06:52:47.644194 2999 tablet_replica.cc:442] T b2e68f788d1640a6aa5d71c7120a8576 P 777f1c96cfae4fcaae963fdc17f98fda: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:52:47.654109 2988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.651264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32838 (local address 127.0.191.194:43509)
0926 06:52:47.651317 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.651319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.651347 (+ 28us) server_negotiation.cc:407] Connection header received
0926 06:52:47.651423 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.651424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.651438 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.651523 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:52:47.651698 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.652162 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.653289 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.653563 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.653625 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.653805 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:52:47.653898 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:52:47.653957 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.654016 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.654035 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 06:52:47.659072 2835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.656428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:42499 (local address 127.0.191.194:38351)
0926 06:52:47.656503 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.656514 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.656569 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.656747 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.656748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.656749 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:52:47.656798 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.656799 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.657683 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.657684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.658237 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.658238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.658700 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.658701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.658756 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.658769 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:52:47.658951 (+ 182us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:52:47.658953 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.658988 (+ 35us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.659027 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 06:52:47.659077 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.656536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:38351 (local address 127.0.191.195:42499)
0926 06:52:47.656611 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.656613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.656623 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:52:47.656643 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.656644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.656654 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.656713 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:52:47.656872 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.657597 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.658309 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.658613 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.658682 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.658862 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:52:47.658893 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:52:47.658945 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.658991 (+ 46us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.659008 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:52:47.661257 2643 raft_consensus.cc:1261] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Refusing update from remote peer 478e14a4b1114d83ab39fcc73f078f3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:47.661343 2923 raft_consensus.cc:1261] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Refusing update from remote peer 478e14a4b1114d83ab39fcc73f078f3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:47.661490 3000 consensus_queue.cc:1035] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:47.661602 3002 consensus_queue.cc:1035] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:47.662886 3021 mvcc.cc:205] Tried to move back new op lower bound from 7333501205138714624 to 7333501204467834880. Current Snapshot: MvccSnapshot[applied={T|T < 7333501205138714624}]
I20260926 06:52:47.663439 3020 mvcc.cc:205] Tried to move back new op lower bound from 7333501205138714624 to 7333501204467834880. Current Snapshot: MvccSnapshot[applied={T|T < 7333501205138714624}]
I20260926 06:52:47.670089 2988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.667708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:41699 (local address 127.0.191.194:43509)
0926 06:52:47.667750 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:47.667751 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:47.667757 (+ 6us) server_negotiation.cc:407] Connection header received
0926 06:52:47.667772 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:47.667773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:47.667781 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:47.667823 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:52:47.667958 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.668372 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.669389 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.669703 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.669757 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.669934 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:52:47.669954 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:52:47.669985 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:47.670035 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 06:52:47.670048 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:52:47.670135 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:47.667504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:43509 (local address 127.0.191.195:41699)
0926 06:52:47.667591 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:47.667601 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:47.667644 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:47.667846 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:47.667846 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:47.667847 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:52:47.667898 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.667900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.668437 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.668438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.669305 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:47.669307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:47.669776 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:47.669777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:47.669845 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:47.669859 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:52:47.669986 (+ 127us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:52:47.669987 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:52:47.670033 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 06:52:47.670068 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 06:52:47.676676 2486 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:52:47.676769 2486 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:52:47.677464 2486 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e: Sending DeleteTablet for 3 replicas of tablet ec41b1d940014456a5a6297cc8493693
I20260926 06:52:47.677760 2763 tablet_service.cc:1514] Processing DeleteTablet for tablet ec41b1d940014456a5a6297cc8493693 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:47 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:32828
I20260926 06:52:47.677882 2623 tablet_service.cc:1514] Processing DeleteTablet for tablet ec41b1d940014456a5a6297cc8493693 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:47 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:45572
I20260926 06:52:47.677917 2903 tablet_service.cc:1514] Processing DeleteTablet for tablet ec41b1d940014456a5a6297cc8493693 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:47 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:36160
I20260926 06:52:47.678262 3038 tablet_replica.cc:330] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda: stopping tablet replica
I20260926 06:52:47.678356 2486 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:52:47.678479 2486 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:52:47.678725 3038 raft_consensus.cc:2229] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:52:47.679513 2903 tablet_service.cc:1467] Processing CreateTablet for tablet 92e2d3c0664843f3825394e9c5e19800 (DEFAULT_TABLE table=test-table [id=6bd94cfba8e747e389342cb889f4b1f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:47.679529 2623 tablet_service.cc:1467] Processing CreateTablet for tablet 92e2d3c0664843f3825394e9c5e19800 (DEFAULT_TABLE table=test-table [id=6bd94cfba8e747e389342cb889f4b1f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:47.679634 2903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92e2d3c0664843f3825394e9c5e19800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:47.679643 2623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92e2d3c0664843f3825394e9c5e19800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:47.680935 2992 tablet_bootstrap.cc:492] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a: Bootstrap starting.
I20260926 06:52:47.681035 2993 tablet_bootstrap.cc:492] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda: Bootstrap starting.
I20260926 06:52:47.681341 2992 tablet_bootstrap.cc:654] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:47.681427 2993 tablet_bootstrap.cc:654] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:47.681995 2763 tablet_service.cc:1467] Processing CreateTablet for tablet 92e2d3c0664843f3825394e9c5e19800 (DEFAULT_TABLE table=test-table [id=6bd94cfba8e747e389342cb889f4b1f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:47.682029 3038 raft_consensus.cc:2258] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:47.682096 2763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92e2d3c0664843f3825394e9c5e19800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:47.682431 3038 ts_tablet_manager.cc:1905] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:47.683380 2995 tablet_bootstrap.cc:492] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e: Bootstrap starting.
I20260926 06:52:47.683805 2995 tablet_bootstrap.cc:654] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:47.684407 3038 ts_tablet_manager.cc:1918] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:52:47.684453 3039 tablet_replica.cc:330] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a: stopping tablet replica
I20260926 06:52:47.684480 3038 log.cc:1196] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/wal/wals/ec41b1d940014456a5a6297cc8493693
I20260926 06:52:47.684547 2995 tablet_bootstrap.cc:492] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e: No bootstrap required, opened a new log
I20260926 06:52:47.684556 3039 raft_consensus.cc:2229] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:52:47.684605 3039 raft_consensus.cc:2258] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:47.684597 2995 ts_tablet_manager.cc:1397] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:52:47.684747 3038 ts_tablet_manager.cc:1939] T ec41b1d940014456a5a6297cc8493693 P 777f1c96cfae4fcaae963fdc17f98fda: Deleting consensus metadata
I20260926 06:52:47.684743 2995 raft_consensus.cc:348] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.684813 2995 raft_consensus.cc:374] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:47.684861 2995 raft_consensus.cc:729] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478e14a4b1114d83ab39fcc73f078f3e, State: Initialized, Role: FOLLOWER
I20260926 06:52:47.684903 2995 consensus_queue.cc:260] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.684989 2995 ts_tablet_manager.cc:1428] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 06:52:47.685109 2470 catalog_manager.cc:4841] TS 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009): tablet ec41b1d940014456a5a6297cc8493693 (table test-table [id=dc9654a21857462c931bfbe5fdaa14fb]) successfully deleted
I20260926 06:52:47.685146 3037 tablet_replica.cc:330] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e: stopping tablet replica
I20260926 06:52:47.685228 3037 raft_consensus.cc:2229] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:52:47.685405 3037 raft_consensus.cc:2258] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:47.685923 3037 ts_tablet_manager.cc:1905] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:47.687536 2993 tablet_bootstrap.cc:492] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda: No bootstrap required, opened a new log
I20260926 06:52:47.687587 2993 ts_tablet_manager.cc:1397] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260926 06:52:47.687706 2992 tablet_bootstrap.cc:492] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a: No bootstrap required, opened a new log
I20260926 06:52:47.687708 2993 raft_consensus.cc:348] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.687750 2992 ts_tablet_manager.cc:1397] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260926 06:52:47.687767 2993 raft_consensus.cc:374] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:47.687788 2993 raft_consensus.cc:729] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 777f1c96cfae4fcaae963fdc17f98fda, State: Initialized, Role: FOLLOWER
I20260926 06:52:47.687835 2993 consensus_queue.cc:260] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.687870 2992 raft_consensus.cc:348] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.687927 2992 raft_consensus.cc:374] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:47.687927 2993 ts_tablet_manager.cc:1428] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:47.687947 2992 raft_consensus.cc:729] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f5854981be45b891c450c46830c20a, State: Initialized, Role: FOLLOWER
I20260926 06:52:47.687987 2992 consensus_queue.cc:260] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.688059 2992 ts_tablet_manager.cc:1428] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:47.688217 3039 ts_tablet_manager.cc:1905] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:47.688217 3037 ts_tablet_manager.cc:1918] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:52:47.688341 3037 log.cc:1196] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/ec41b1d940014456a5a6297cc8493693
I20260926 06:52:47.688676 3037 ts_tablet_manager.cc:1939] T ec41b1d940014456a5a6297cc8493693 P 478e14a4b1114d83ab39fcc73f078f3e: Deleting consensus metadata
I20260926 06:52:47.689057 2470 catalog_manager.cc:4841] TS 478e14a4b1114d83ab39fcc73f078f3e (127.0.191.194:43509): tablet ec41b1d940014456a5a6297cc8493693 (table test-table [id=dc9654a21857462c931bfbe5fdaa14fb]) successfully deleted
I20260926 06:52:47.689895 3039 ts_tablet_manager.cc:1918] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:52:47.689966 3039 log.cc:1196] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/ec41b1d940014456a5a6297cc8493693
I20260926 06:52:47.690208 3039 ts_tablet_manager.cc:1939] T ec41b1d940014456a5a6297cc8493693 P a0f5854981be45b891c450c46830c20a: Deleting consensus metadata
I20260926 06:52:47.690593 2473 catalog_manager.cc:4841] TS a0f5854981be45b891c450c46830c20a (127.0.191.195:42499): tablet ec41b1d940014456a5a6297cc8493693 (table test-table [id=dc9654a21857462c931bfbe5fdaa14fb]) successfully deleted
I20260926 06:52:47.788686 3033 raft_consensus.cc:482] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:52:47.788781 3033 raft_consensus.cc:504] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.788954 3033 leader_election.cc:288] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009), a0f5854981be45b891c450c46830c20a (127.0.191.195:42499)
I20260926 06:52:47.789145 2643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92e2d3c0664843f3825394e9c5e19800" candidate_uuid: "478e14a4b1114d83ab39fcc73f078f3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777f1c96cfae4fcaae963fdc17f98fda" is_pre_election: true
I20260926 06:52:47.789166 2923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92e2d3c0664843f3825394e9c5e19800" candidate_uuid: "478e14a4b1114d83ab39fcc73f078f3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0f5854981be45b891c450c46830c20a" is_pre_election: true
I20260926 06:52:47.789247 2643 raft_consensus.cc:2454] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478e14a4b1114d83ab39fcc73f078f3e in term 0.
I20260926 06:52:47.789270 2923 raft_consensus.cc:2454] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478e14a4b1114d83ab39fcc73f078f3e in term 0.
I20260926 06:52:47.789419 2714 leader_election.cc:302] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478e14a4b1114d83ab39fcc73f078f3e, 777f1c96cfae4fcaae963fdc17f98fda; no voters:
I20260926 06:52:47.789494 3033 raft_consensus.cc:2781] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:52:47.789520 3033 raft_consensus.cc:482] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:52:47.789542 3033 raft_consensus.cc:3037] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:47.790362 3033 raft_consensus.cc:504] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.790482 3033 leader_election.cc:288] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [CANDIDATE]: Term 1 election: Requested vote from peers 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009), a0f5854981be45b891c450c46830c20a (127.0.191.195:42499)
I20260926 06:52:47.790632 2923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92e2d3c0664843f3825394e9c5e19800" candidate_uuid: "478e14a4b1114d83ab39fcc73f078f3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0f5854981be45b891c450c46830c20a"
I20260926 06:52:47.790704 2923 raft_consensus.cc:3037] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:47.790655 2643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92e2d3c0664843f3825394e9c5e19800" candidate_uuid: "478e14a4b1114d83ab39fcc73f078f3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777f1c96cfae4fcaae963fdc17f98fda"
I20260926 06:52:47.790735 2643 raft_consensus.cc:3037] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:47.791392 2643 raft_consensus.cc:2454] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478e14a4b1114d83ab39fcc73f078f3e in term 1.
I20260926 06:52:47.791445 2923 raft_consensus.cc:2454] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478e14a4b1114d83ab39fcc73f078f3e in term 1.
I20260926 06:52:47.791567 2714 leader_election.cc:302] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478e14a4b1114d83ab39fcc73f078f3e, 777f1c96cfae4fcaae963fdc17f98fda; no voters:
I20260926 06:52:47.791653 3033 raft_consensus.cc:2781] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:47.791704 3033 raft_consensus.cc:686] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 LEADER]: Becoming Leader. State: Replica: 478e14a4b1114d83ab39fcc73f078f3e, State: Running, Role: LEADER
I20260926 06:52:47.791769 3033 consensus_queue.cc:237] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:47.792403 2486 catalog_manager.cc:5495] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 478e14a4b1114d83ab39fcc73f078f3e (127.0.191.194). New cstate: current_term: 1 leader_uuid: "478e14a4b1114d83ab39fcc73f078f3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260926 06:52:47.803198 2923 raft_consensus.cc:1261] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Refusing update from remote peer 478e14a4b1114d83ab39fcc73f078f3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:47.803222 2643 raft_consensus.cc:1261] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Refusing update from remote peer 478e14a4b1114d83ab39fcc73f078f3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:47.803412 3000 consensus_queue.cc:1035] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:47.803503 3033 consensus_queue.cc:1035] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:47.804864 3019 mvcc.cc:205] Tried to move back new op lower bound from 7333501205720719360 to 7333501205675634688. Current Snapshot: MvccSnapshot[applied={T|T < 7333501205720719360}]
W20260926 06:52:47.807875 3021 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 06:52:49.324671 2486 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:52:49.324770 2486 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:52:49.325630 2486 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e: Sending DeleteTablet for 3 replicas of tablet 92e2d3c0664843f3825394e9c5e19800
I20260926 06:52:49.325932 2763 tablet_service.cc:1514] Processing DeleteTablet for tablet 92e2d3c0664843f3825394e9c5e19800 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:49 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:32828
I20260926 06:52:49.325984 2903 tablet_service.cc:1514] Processing DeleteTablet for tablet 92e2d3c0664843f3825394e9c5e19800 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:49 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:36160
I20260926 06:52:49.325984 2623 tablet_service.cc:1514] Processing DeleteTablet for tablet 92e2d3c0664843f3825394e9c5e19800 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:49 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:45572
I20260926 06:52:49.328222 3069 tablet_replica.cc:330] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a: stopping tablet replica
I20260926 06:52:49.328318 3069 raft_consensus.cc:2229] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:52:49.328358 3069 raft_consensus.cc:2258] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:49.328457 3070 tablet_replica.cc:330] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda: stopping tablet replica
I20260926 06:52:49.328516 3070 raft_consensus.cc:2229] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:52:49.328552 3070 raft_consensus.cc:2258] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:49.328562 3069 ts_tablet_manager.cc:1905] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:49.328890 3070 ts_tablet_manager.cc:1905] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:49.329489 3068 tablet_replica.cc:330] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e: stopping tablet replica
I20260926 06:52:49.329561 3068 raft_consensus.cc:2229] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:52:49.329644 3068 raft_consensus.cc:2258] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:49.330456 3069 ts_tablet_manager.cc:1918] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 06:52:49.330546 3069 log.cc:1196] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/92e2d3c0664843f3825394e9c5e19800
I20260926 06:52:49.330631 3070 ts_tablet_manager.cc:1918] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 06:52:49.330745 3070 log.cc:1196] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/wal/wals/92e2d3c0664843f3825394e9c5e19800
I20260926 06:52:49.330809 3069 ts_tablet_manager.cc:1939] T 92e2d3c0664843f3825394e9c5e19800 P a0f5854981be45b891c450c46830c20a: Deleting consensus metadata
I20260926 06:52:49.330978 3070 ts_tablet_manager.cc:1939] T 92e2d3c0664843f3825394e9c5e19800 P 777f1c96cfae4fcaae963fdc17f98fda: Deleting consensus metadata
I20260926 06:52:49.331065 3068 ts_tablet_manager.cc:1905] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:49.331288 2473 catalog_manager.cc:4841] TS a0f5854981be45b891c450c46830c20a (127.0.191.195:42499): tablet 92e2d3c0664843f3825394e9c5e19800 (table test-table [id=6bd94cfba8e747e389342cb889f4b1f1]) successfully deleted
I20260926 06:52:49.331374 2470 catalog_manager.cc:4841] TS 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009): tablet 92e2d3c0664843f3825394e9c5e19800 (table test-table [id=6bd94cfba8e747e389342cb889f4b1f1]) successfully deleted
I20260926 06:52:49.332477 3068 ts_tablet_manager.cc:1918] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 06:52:49.332525 3068 log.cc:1196] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/92e2d3c0664843f3825394e9c5e19800
I20260926 06:52:49.332712 3068 ts_tablet_manager.cc:1939] T 92e2d3c0664843f3825394e9c5e19800 P 478e14a4b1114d83ab39fcc73f078f3e: Deleting consensus metadata
I20260926 06:52:49.332974 2470 catalog_manager.cc:4841] TS 478e14a4b1114d83ab39fcc73f078f3e (127.0.191.194:43509): tablet 92e2d3c0664843f3825394e9c5e19800 (table test-table [id=6bd94cfba8e747e389342cb889f4b1f1]) successfully deleted
Sep 26 06:52:49 dist-test-slave-ws74 krb5kdc[2441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405569, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 06:52:49 dist-test-slave-ws74 krb5kdc[2441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405569, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:49.355993 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:49.344571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50858 (local address 127.0.191.254:46729)
0926 06:52:49.344701 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:49.344703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:49.344715 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:52:49.344815 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:49.344816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:49.344828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:49.344894 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:49.345471 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:49.345980 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:49.346693 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:49.346913 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:49.346996 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:49.353835 (+ 6839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:49.353837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:49.353837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:49.353844 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:49.355077 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:49.355373 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:49.355374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:49.355374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:49.355422 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:49.355578 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:49.355579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:49.355579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:49.355691 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:49.355765 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:49.355863 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 06:52:49.355923 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260926 06:52:49.357524 2486 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:52:49.357692 2486 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:52:49.359082 2623 tablet_service.cc:1467] Processing CreateTablet for tablet 388e84c05775463e87664a06f2986bd1 (DEFAULT_TABLE table=test-table [id=b2116ad548774d059b0e63170220ba08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:49.359098 2763 tablet_service.cc:1467] Processing CreateTablet for tablet 388e84c05775463e87664a06f2986bd1 (DEFAULT_TABLE table=test-table [id=b2116ad548774d059b0e63170220ba08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:49.359232 2763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388e84c05775463e87664a06f2986bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:49.359233 2623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388e84c05775463e87664a06f2986bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:49.360193 3079 tablet_bootstrap.cc:492] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda: Bootstrap starting.
I20260926 06:52:49.360208 3080 tablet_bootstrap.cc:492] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e: Bootstrap starting.
I20260926 06:52:49.360261 2903 tablet_service.cc:1467] Processing CreateTablet for tablet 388e84c05775463e87664a06f2986bd1 (DEFAULT_TABLE table=test-table [id=b2116ad548774d059b0e63170220ba08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:49.360378 2903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388e84c05775463e87664a06f2986bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:49.360621 3079 tablet_bootstrap.cc:654] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:49.360651 3080 tablet_bootstrap.cc:654] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:49.361204 3079 tablet_bootstrap.cc:492] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda: No bootstrap required, opened a new log
I20260926 06:52:49.361274 3079 ts_tablet_manager.cc:1397] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:52:49.361466 3079 raft_consensus.cc:348] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.361545 3079 raft_consensus.cc:374] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:49.361568 3079 raft_consensus.cc:729] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 777f1c96cfae4fcaae963fdc17f98fda, State: Initialized, Role: FOLLOWER
I20260926 06:52:49.361615 3079 consensus_queue.cc:260] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.361779 3083 tablet_bootstrap.cc:492] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a: Bootstrap starting.
I20260926 06:52:49.361790 3079 ts_tablet_manager.cc:1428] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 06:52:49.361990 3080 tablet_bootstrap.cc:492] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e: No bootstrap required, opened a new log
I20260926 06:52:49.362036 3080 ts_tablet_manager.cc:1397] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:52:49.362133 3080 raft_consensus.cc:348] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.362175 3083 tablet_bootstrap.cc:654] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:49.362221 3080 raft_consensus.cc:374] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:49.362246 3080 raft_consensus.cc:729] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478e14a4b1114d83ab39fcc73f078f3e, State: Initialized, Role: FOLLOWER
I20260926 06:52:49.362298 3080 consensus_queue.cc:260] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.362390 3080 ts_tablet_manager.cc:1428] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:49.363035 3083 tablet_bootstrap.cc:492] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a: No bootstrap required, opened a new log
I20260926 06:52:49.363090 3083 ts_tablet_manager.cc:1397] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:52:49.363197 3083 raft_consensus.cc:348] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.363256 3083 raft_consensus.cc:374] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:49.363269 3083 raft_consensus.cc:729] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f5854981be45b891c450c46830c20a, State: Initialized, Role: FOLLOWER
I20260926 06:52:49.363323 3083 consensus_queue.cc:260] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.363415 3083 ts_tablet_manager.cc:1428] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:49.477118 3049 raft_consensus.cc:482] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:52:49.477214 3049 raft_consensus.cc:504] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.477370 3049 leader_election.cc:288] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009), 478e14a4b1114d83ab39fcc73f078f3e (127.0.191.194:43509)
I20260926 06:52:49.477612 2783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388e84c05775463e87664a06f2986bd1" candidate_uuid: "a0f5854981be45b891c450c46830c20a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478e14a4b1114d83ab39fcc73f078f3e" is_pre_election: true
I20260926 06:52:49.477625 2642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388e84c05775463e87664a06f2986bd1" candidate_uuid: "a0f5854981be45b891c450c46830c20a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777f1c96cfae4fcaae963fdc17f98fda" is_pre_election: true
I20260926 06:52:49.477720 2642 raft_consensus.cc:2454] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f5854981be45b891c450c46830c20a in term 0.
I20260926 06:52:49.477731 2783 raft_consensus.cc:2454] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f5854981be45b891c450c46830c20a in term 0.
I20260926 06:52:49.477880 2854 leader_election.cc:302] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478e14a4b1114d83ab39fcc73f078f3e, a0f5854981be45b891c450c46830c20a; no voters:
I20260926 06:52:49.478016 3049 raft_consensus.cc:2781] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:52:49.478070 3049 raft_consensus.cc:482] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:52:49.478085 3049 raft_consensus.cc:3037] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:49.478782 3049 raft_consensus.cc:504] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.478905 3049 leader_election.cc:288] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [CANDIDATE]: Term 1 election: Requested vote from peers 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009), 478e14a4b1114d83ab39fcc73f078f3e (127.0.191.194:43509)
I20260926 06:52:49.479096 2642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388e84c05775463e87664a06f2986bd1" candidate_uuid: "a0f5854981be45b891c450c46830c20a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777f1c96cfae4fcaae963fdc17f98fda"
I20260926 06:52:49.479168 2642 raft_consensus.cc:3037] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:49.479135 2783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388e84c05775463e87664a06f2986bd1" candidate_uuid: "a0f5854981be45b891c450c46830c20a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478e14a4b1114d83ab39fcc73f078f3e"
I20260926 06:52:49.479215 2783 raft_consensus.cc:3037] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:49.479781 2642 raft_consensus.cc:2454] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f5854981be45b891c450c46830c20a in term 1.
I20260926 06:52:49.479903 2783 raft_consensus.cc:2454] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f5854981be45b891c450c46830c20a in term 1.
I20260926 06:52:49.479933 2854 leader_election.cc:302] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 777f1c96cfae4fcaae963fdc17f98fda, a0f5854981be45b891c450c46830c20a; no voters:
I20260926 06:52:49.480011 3049 raft_consensus.cc:2781] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:49.480067 3049 raft_consensus.cc:686] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [term 1 LEADER]: Becoming Leader. State: Replica: a0f5854981be45b891c450c46830c20a, State: Running, Role: LEADER
I20260926 06:52:49.480124 3049 consensus_queue.cc:237] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.480767 2485 catalog_manager.cc:5495] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a reported cstate change: term changed from 0 to 1, leader changed from <none> to a0f5854981be45b891c450c46830c20a (127.0.191.195). New cstate: current_term: 1 leader_uuid: "a0f5854981be45b891c450c46830c20a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260926 06:52:49.511950 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:49.508903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36176 (local address 127.0.191.195:42499)
0926 06:52:49.509474 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:49.509476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:49.509484 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:52:49.509510 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:49.509511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:49.509520 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:49.509601 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:52:49.509755 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:49.510319 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:49.511231 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:49.511451 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:49.511528 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:49.511670 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:52:49.511705 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:52:49.511777 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:49.511841 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 06:52:49.511861 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":87,"threads_started":1}
I20260926 06:52:49.515827 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:49.513931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:38673 (local address 127.0.191.195:42499)
0926 06:52:49.513983 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:49.513985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:49.514041 (+ 56us) server_negotiation.cc:407] Connection header received
0926 06:52:49.514085 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:49.514086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:49.514095 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:49.514137 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:52:49.514258 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:49.514667 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:49.515339 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:49.515504 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:49.515551 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:49.515650 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:52:49.515670 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:52:49.515703 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:49.515750 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 06:52:49.515764 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:52:49.515889 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:49.513853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:42499 (local address 127.0.191.195:38673)
0926 06:52:49.513994 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:49.514003 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:49.514050 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:49.514172 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:49.514172 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:49.514174 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:52:49.514208 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:49.514209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:49.514735 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:49.514735 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:49.515276 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:49.515277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:49.515563 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:49.515564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:49.515600 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:49.515603 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:52:49.515716 (+ 113us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:52:49.515717 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:52:49.515750 (+ 33us) client_negotiation.cc:240] Negotiation successful
0926 06:52:49.515784 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
I20260926 06:52:49.517882 2783 raft_consensus.cc:1261] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Refusing update from remote peer a0f5854981be45b891c450c46830c20a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:49.517915 2642 raft_consensus.cc:1261] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Refusing update from remote peer a0f5854981be45b891c450c46830c20a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:49.518085 3046 consensus_queue.cc:1035] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [LEADER]: Connected to new peer: Peer: permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:49.518172 3049 consensus_queue.cc:1035] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [LEADER]: Connected to new peer: Peer: permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:49.528034 2485 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:52:49.528128 2485 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:52:49.528692 2485 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e: Sending DeleteTablet for 3 replicas of tablet 388e84c05775463e87664a06f2986bd1
I20260926 06:52:49.529000 2763 tablet_service.cc:1514] Processing DeleteTablet for tablet 388e84c05775463e87664a06f2986bd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:49 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:32828
I20260926 06:52:49.529006 2623 tablet_service.cc:1514] Processing DeleteTablet for tablet 388e84c05775463e87664a06f2986bd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:49 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:45572
I20260926 06:52:49.529093 3070 tablet_replica.cc:330] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda: stopping tablet replica
I20260926 06:52:49.529093 3068 tablet_replica.cc:330] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e: stopping tablet replica
I20260926 06:52:49.529109 2903 tablet_service.cc:1514] Processing DeleteTablet for tablet 388e84c05775463e87664a06f2986bd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:49 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:36160
I20260926 06:52:49.529137 3068 raft_consensus.cc:2229] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:52:49.529134 3070 raft_consensus.cc:2229] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:52:49.529175 3068 raft_consensus.cc:2258] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:49.529177 3070 raft_consensus.cc:2258] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:49.529206 3069 tablet_replica.cc:330] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a: stopping tablet replica
I20260926 06:52:49.529269 3069 raft_consensus.cc:2229] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:52:49.529352 3069 raft_consensus.cc:2258] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:49.529587 3068 ts_tablet_manager.cc:1905] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:49.529667 3069 ts_tablet_manager.cc:1905] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:49.531379 3068 ts_tablet_manager.cc:1918] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:52:49.531409 3069 ts_tablet_manager.cc:1918] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:52:49.531402 3070 ts_tablet_manager.cc:1905] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:49.531436 3068 log.cc:1196] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/388e84c05775463e87664a06f2986bd1
I20260926 06:52:49.531455 3069 log.cc:1196] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/388e84c05775463e87664a06f2986bd1
I20260926 06:52:49.531654 3068 ts_tablet_manager.cc:1939] T 388e84c05775463e87664a06f2986bd1 P 478e14a4b1114d83ab39fcc73f078f3e: Deleting consensus metadata
I20260926 06:52:49.531666 3069 ts_tablet_manager.cc:1939] T 388e84c05775463e87664a06f2986bd1 P a0f5854981be45b891c450c46830c20a: Deleting consensus metadata
I20260926 06:52:49.532635 2470 catalog_manager.cc:4841] TS 478e14a4b1114d83ab39fcc73f078f3e (127.0.191.194:43509): tablet 388e84c05775463e87664a06f2986bd1 (table test-table [id=b2116ad548774d059b0e63170220ba08]) successfully deleted
I20260926 06:52:49.532720 2473 catalog_manager.cc:4841] TS a0f5854981be45b891c450c46830c20a (127.0.191.195:42499): tablet 388e84c05775463e87664a06f2986bd1 (table test-table [id=b2116ad548774d059b0e63170220ba08]) successfully deleted
I20260926 06:52:49.532807 2485 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:52:49.532905 2485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:52:49.533102 3070 ts_tablet_manager.cc:1918] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:52:49.533155 3070 log.cc:1196] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-0/wal/wals/388e84c05775463e87664a06f2986bd1
I20260926 06:52:49.533383 3070 ts_tablet_manager.cc:1939] T 388e84c05775463e87664a06f2986bd1 P 777f1c96cfae4fcaae963fdc17f98fda: Deleting consensus metadata
I20260926 06:52:49.533695 2470 catalog_manager.cc:4841] TS 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009): tablet 388e84c05775463e87664a06f2986bd1 (table test-table [id=b2116ad548774d059b0e63170220ba08]) successfully deleted
I20260926 06:52:49.534277 2903 tablet_service.cc:1467] Processing CreateTablet for tablet ad3b7224ff23426c8c252ff4b4eb6410 (DEFAULT_TABLE table=test-table [id=ab1f72c78e104ddd95deadc19db9a062]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:49.534289 2763 tablet_service.cc:1467] Processing CreateTablet for tablet ad3b7224ff23426c8c252ff4b4eb6410 (DEFAULT_TABLE table=test-table [id=ab1f72c78e104ddd95deadc19db9a062]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:49.534281 2623 tablet_service.cc:1467] Processing CreateTablet for tablet ad3b7224ff23426c8c252ff4b4eb6410 (DEFAULT_TABLE table=test-table [id=ab1f72c78e104ddd95deadc19db9a062]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:52:49.534379 2623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad3b7224ff23426c8c252ff4b4eb6410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:49.534380 2903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad3b7224ff23426c8c252ff4b4eb6410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:49.534382 2763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad3b7224ff23426c8c252ff4b4eb6410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:49.535460 3080 tablet_bootstrap.cc:492] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e: Bootstrap starting.
I20260926 06:52:49.535499 3083 tablet_bootstrap.cc:492] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a: Bootstrap starting.
I20260926 06:52:49.535737 3080 tablet_bootstrap.cc:654] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:49.535784 3083 tablet_bootstrap.cc:654] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:49.535945 3079 tablet_bootstrap.cc:492] T ad3b7224ff23426c8c252ff4b4eb6410 P 777f1c96cfae4fcaae963fdc17f98fda: Bootstrap starting.
I20260926 06:52:49.536291 3079 tablet_bootstrap.cc:654] T ad3b7224ff23426c8c252ff4b4eb6410 P 777f1c96cfae4fcaae963fdc17f98fda: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:49.536556 3083 tablet_bootstrap.cc:492] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a: No bootstrap required, opened a new log
I20260926 06:52:49.536602 3083 ts_tablet_manager.cc:1397] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:52:49.536715 3083 raft_consensus.cc:348] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.536754 3080 tablet_bootstrap.cc:492] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e: No bootstrap required, opened a new log
I20260926 06:52:49.536772 3083 raft_consensus.cc:374] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:49.536787 3080 ts_tablet_manager.cc:1397] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:52:49.536792 3083 raft_consensus.cc:729] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f5854981be45b891c450c46830c20a, State: Initialized, Role: FOLLOWER
I20260926 06:52:49.536836 3083 consensus_queue.cc:260] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.536914 3083 ts_tablet_manager.cc:1428] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:49.536891 3080 raft_consensus.cc:348] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.536953 3080 raft_consensus.cc:374] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:49.536972 3080 raft_consensus.cc:729] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478e14a4b1114d83ab39fcc73f078f3e, State: Initialized, Role: FOLLOWER
I20260926 06:52:49.537014 3080 consensus_queue.cc:260] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.537117 3080 ts_tablet_manager.cc:1428] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:49.537487 3079 tablet_bootstrap.cc:492] T ad3b7224ff23426c8c252ff4b4eb6410 P 777f1c96cfae4fcaae963fdc17f98fda: No bootstrap required, opened a new log
I20260926 06:52:49.537530 3079 ts_tablet_manager.cc:1397] T ad3b7224ff23426c8c252ff4b4eb6410 P 777f1c96cfae4fcaae963fdc17f98fda: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:52:49.537660 3079 raft_consensus.cc:348] T ad3b7224ff23426c8c252ff4b4eb6410 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.537721 3079 raft_consensus.cc:374] T ad3b7224ff23426c8c252ff4b4eb6410 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:49.537735 3079 raft_consensus.cc:729] T ad3b7224ff23426c8c252ff4b4eb6410 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 777f1c96cfae4fcaae963fdc17f98fda, State: Initialized, Role: FOLLOWER
I20260926 06:52:49.537786 3079 consensus_queue.cc:260] T ad3b7224ff23426c8c252ff4b4eb6410 P 777f1c96cfae4fcaae963fdc17f98fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.537873 3079 ts_tablet_manager.cc:1428] T ad3b7224ff23426c8c252ff4b4eb6410 P 777f1c96cfae4fcaae963fdc17f98fda: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:49.598678 3063 raft_consensus.cc:482] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:52:49.598767 3063 raft_consensus.cc:504] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.598948 3063 leader_election.cc:288] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009), a0f5854981be45b891c450c46830c20a (127.0.191.195:42499)
I20260926 06:52:49.599115 2923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad3b7224ff23426c8c252ff4b4eb6410" candidate_uuid: "478e14a4b1114d83ab39fcc73f078f3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0f5854981be45b891c450c46830c20a" is_pre_election: true
I20260926 06:52:49.599115 2643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad3b7224ff23426c8c252ff4b4eb6410" candidate_uuid: "478e14a4b1114d83ab39fcc73f078f3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777f1c96cfae4fcaae963fdc17f98fda" is_pre_election: true
I20260926 06:52:49.599222 2923 raft_consensus.cc:2454] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478e14a4b1114d83ab39fcc73f078f3e in term 0.
I20260926 06:52:49.599229 2643 raft_consensus.cc:2454] T ad3b7224ff23426c8c252ff4b4eb6410 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478e14a4b1114d83ab39fcc73f078f3e in term 0.
I20260926 06:52:49.599433 2714 leader_election.cc:302] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478e14a4b1114d83ab39fcc73f078f3e, 777f1c96cfae4fcaae963fdc17f98fda; no voters:
I20260926 06:52:49.599546 3063 raft_consensus.cc:2781] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:52:49.599592 3063 raft_consensus.cc:482] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:52:49.599620 3063 raft_consensus.cc:3037] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:49.600391 3063 raft_consensus.cc:504] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.600528 3063 leader_election.cc:288] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [CANDIDATE]: Term 1 election: Requested vote from peers 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009), a0f5854981be45b891c450c46830c20a (127.0.191.195:42499)
I20260926 06:52:49.600673 2643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad3b7224ff23426c8c252ff4b4eb6410" candidate_uuid: "478e14a4b1114d83ab39fcc73f078f3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777f1c96cfae4fcaae963fdc17f98fda"
I20260926 06:52:49.600718 2923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad3b7224ff23426c8c252ff4b4eb6410" candidate_uuid: "478e14a4b1114d83ab39fcc73f078f3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0f5854981be45b891c450c46830c20a"
I20260926 06:52:49.600759 2643 raft_consensus.cc:3037] T ad3b7224ff23426c8c252ff4b4eb6410 P 777f1c96cfae4fcaae963fdc17f98fda [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:49.600775 2923 raft_consensus.cc:3037] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:49.601488 2643 raft_consensus.cc:2454] T ad3b7224ff23426c8c252ff4b4eb6410 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478e14a4b1114d83ab39fcc73f078f3e in term 1.
I20260926 06:52:49.601488 2923 raft_consensus.cc:2454] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478e14a4b1114d83ab39fcc73f078f3e in term 1.
I20260926 06:52:49.601649 2714 leader_election.cc:302] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478e14a4b1114d83ab39fcc73f078f3e, 777f1c96cfae4fcaae963fdc17f98fda; no voters:
I20260926 06:52:49.601706 3063 raft_consensus.cc:2781] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:49.601763 3063 raft_consensus.cc:686] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 LEADER]: Becoming Leader. State: Replica: 478e14a4b1114d83ab39fcc73f078f3e, State: Running, Role: LEADER
I20260926 06:52:49.601807 3063 consensus_queue.cc:237] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } }
I20260926 06:52:49.602437 2485 catalog_manager.cc:5495] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 478e14a4b1114d83ab39fcc73f078f3e (127.0.191.194). New cstate: current_term: 1 leader_uuid: "478e14a4b1114d83ab39fcc73f078f3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478e14a4b1114d83ab39fcc73f078f3e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260926 06:52:49.616549 3100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:49.614187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32842 (local address 127.0.191.194:43509)
0926 06:52:49.614362 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:49.614365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:49.614404 (+ 39us) server_negotiation.cc:407] Connection header received
0926 06:52:49.614468 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:49.614469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:49.614480 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:49.614546 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:52:49.614645 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:49.615113 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:49.615806 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:49.616034 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:49.616117 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:49.616270 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:52:49.616323 (+ 53us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:52:49.616380 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:49.616437 (+ 57us) server_negotiation.cc:299] Negotiation successful
0926 06:52:49.616459 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":128,"threads_started":1}
I20260926 06:52:49.618656 2923 raft_consensus.cc:1261] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Refusing update from remote peer 478e14a4b1114d83ab39fcc73f078f3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:49.618695 2643 raft_consensus.cc:1261] T ad3b7224ff23426c8c252ff4b4eb6410 P 777f1c96cfae4fcaae963fdc17f98fda [term 1 FOLLOWER]: Refusing update from remote peer 478e14a4b1114d83ab39fcc73f078f3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:52:49.618852 3059 consensus_queue.cc:1035] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0f5854981be45b891c450c46830c20a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:52:49.618923 3063 consensus_queue.cc:1035] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "777f1c96cfae4fcaae963fdc17f98fda" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 06:52:49.623591 3050 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 06:52:51.141340 2484 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:52:51.141489 2484 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:52:51.142204 2484 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7a113f8ae3e49199f64bbe6587b475e: Sending DeleteTablet for 3 replicas of tablet ad3b7224ff23426c8c252ff4b4eb6410
I20260926 06:52:51.142449 2763 tablet_service.cc:1514] Processing DeleteTablet for tablet ad3b7224ff23426c8c252ff4b4eb6410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:51 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:32828
I20260926 06:52:51.142477 2623 tablet_service.cc:1514] Processing DeleteTablet for tablet ad3b7224ff23426c8c252ff4b4eb6410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:51 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:45572
I20260926 06:52:51.142475 2903 tablet_service.cc:1514] Processing DeleteTablet for tablet ad3b7224ff23426c8c252ff4b4eb6410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:52:51 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:36160
I20260926 06:52:51.142611 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 2558
I20260926 06:52:51.143205 3127 tablet_replica.cc:330] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a: stopping tablet replica
I20260926 06:52:51.143285 3127 raft_consensus.cc:2229] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:52:51.143332 3127 raft_consensus.cc:2258] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:51.143567 3127 ts_tablet_manager.cc:1905] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:51.145541 3127 ts_tablet_manager.cc:1918] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 06:52:51.145604 3127 log.cc:1196] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/ad3b7224ff23426c8c252ff4b4eb6410
I20260926 06:52:51.145850 3127 ts_tablet_manager.cc:1939] T ad3b7224ff23426c8c252ff4b4eb6410 P a0f5854981be45b891c450c46830c20a: Deleting consensus metadata
I20260926 06:52:51.146013 3125 tablet_replica.cc:330] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e: stopping tablet replica
I20260926 06:52:51.146080 3125 raft_consensus.cc:2229] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:52:51.146160 3125 raft_consensus.cc:2258] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:52:51.146421 2473 catalog_manager.cc:4841] TS a0f5854981be45b891c450c46830c20a (127.0.191.195:42499): tablet ad3b7224ff23426c8c252ff4b4eb6410 (table test-table [id=ab1f72c78e104ddd95deadc19db9a062]) successfully deleted
I20260926 06:52:51.146471 3125 ts_tablet_manager.cc:1905] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:52:51.148285 3125 ts_tablet_manager.cc:1918] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 06:52:51.148350 3125 log.cc:1196] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/ad3b7224ff23426c8c252ff4b4eb6410
I20260926 06:52:51.148581 3125 ts_tablet_manager.cc:1939] T ad3b7224ff23426c8c252ff4b4eb6410 P 478e14a4b1114d83ab39fcc73f078f3e: Deleting consensus metadata
I20260926 06:52:51.148914 2470 catalog_manager.cc:4841] TS 478e14a4b1114d83ab39fcc73f078f3e (127.0.191.194:43509): tablet ad3b7224ff23426c8c252ff4b4eb6410 (table test-table [id=ab1f72c78e104ddd95deadc19db9a062]) successfully deleted
W20260926 06:52:51.150249 2470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:39009 (error 108)
I20260926 06:52:51.150820 3129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:51.150614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39009 (local address 127.0.0.1:51638)
0926 06:52:51.150729 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:51.150777 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":56,"threads_started":1}
I20260926 06:52:51.150882 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 2698
W20260926 06:52:51.150933 2470 catalog_manager.cc:4584] TS 777f1c96cfae4fcaae963fdc17f98fda (127.0.191.193:39009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad3b7224ff23426c8c252ff4b4eb6410: Network error: Client connection negotiation failed: client connection to 127.0.191.193:39009: connect: Connection refused (error 111)
I20260926 06:52:51.156464 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 2838
I20260926 06:52:51.162256 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 2454
2026-09-26T06:52:51Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5613 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:52:51 dist-test-slave-ws74 krb5kdc[3132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:52:51 dist-test-slave-ws74 krb5kdc[3132](info): set up 2 sockets
Sep 26 06:52:51 dist-test-slave-ws74 krb5kdc[3132](info): commencing operation
krb5kdc: starting...
W20260926 06:52:52.193079 767 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:52:52 dist-test-slave-ws74 krb5kdc[3132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:52:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:52:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:52:52.304765 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:44769
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:44769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:52.383375 3145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:52.383571 3145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:52.383594 3145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:52.384778 3145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:52:52.384819 3145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:52.384833 3145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:52.384846 3145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:52:52.384858 3145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:52:52.386353 3145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:44769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:44769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:52.386592 3145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:52.386819 3145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:52:52.391541 3145 server_base.cc:1034] running on GCE node
W20260926 06:52:52.391490 3150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:52.391498 3153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:52.391507 3151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:52.392017 3145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:52.392297 3145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:52.393442 3145 hybrid_clock.cc:648] HybridClock initialized: now 1790405572393425 us; error 27 us; skew 500 ppm
Sep 26 06:52:52 dist-test-slave-ws74 krb5kdc[3132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405572, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:52.395747 3145 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:52:52.396198 3145 webserver.cc:460] Webserver started at http://127.0.191.254:44119/ using document root <none> and password file <none>
I20260926 06:52:52.396440 3145 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:52.396505 3145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:52.396643 3145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:52.397715 3145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "bcabb0f147cf4254a6bd073c4802c00c"
format_stamp: "Formatted at 2026-09-26 06:52:52 on dist-test-slave-ws74"
server_key: "d4fa0ceed711b6ff86c88f4741da2121"
server_key_iv: "c1d72864cd50a71ed9515183bba1e5e1"
server_key_version: "encryptionkey@0"
I20260926 06:52:52.398056 3145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "bcabb0f147cf4254a6bd073c4802c00c"
format_stamp: "Formatted at 2026-09-26 06:52:52 on dist-test-slave-ws74"
server_key: "d4fa0ceed711b6ff86c88f4741da2121"
server_key_iv: "c1d72864cd50a71ed9515183bba1e5e1"
server_key_version: "encryptionkey@0"
I20260926 06:52:52.399295 3145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:52:52.399955 3160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:52.400123 3145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:52:52.400218 3145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "bcabb0f147cf4254a6bd073c4802c00c"
format_stamp: "Formatted at 2026-09-26 06:52:52 on dist-test-slave-ws74"
server_key: "d4fa0ceed711b6ff86c88f4741da2121"
server_key_iv: "c1d72864cd50a71ed9515183bba1e5e1"
server_key_version: "encryptionkey@0"
I20260926 06:52:52.400297 3145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:52.413247 3145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:52.414970 3145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:52.415146 3145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:52.419091 3145 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:44769
I20260926 06:52:52.419121 3212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:44769 every 8 connection(s)
I20260926 06:52:52.419608 3145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:52:52.420204 3213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:52.422410 3213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c: Bootstrap starting.
I20260926 06:52:52.422986 3213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:52.423300 3213 log.cc:824] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:52.423949 3213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c: No bootstrap required, opened a new log
I20260926 06:52:52.425489 3213 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcabb0f147cf4254a6bd073c4802c00c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44769 } }
I20260926 06:52:52.425617 3213 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:52.425662 3213 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcabb0f147cf4254a6bd073c4802c00c, State: Initialized, Role: FOLLOWER
I20260926 06:52:52.425822 3213 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcabb0f147cf4254a6bd073c4802c00c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44769 } }
I20260926 06:52:52.425899 3213 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:52:52.425941 3213 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:52:52.425989 3213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:52.426833 3213 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcabb0f147cf4254a6bd073c4802c00c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44769 } }
I20260926 06:52:52.426939 3213 leader_election.cc:302] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcabb0f147cf4254a6bd073c4802c00c; no voters:
I20260926 06:52:52.427122 3213 leader_election.cc:288] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:52:52.427165 3216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:52.427307 3216 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [term 1 LEADER]: Becoming Leader. State: Replica: bcabb0f147cf4254a6bd073c4802c00c, State: Running, Role: LEADER
I20260926 06:52:52.427387 3213 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:52:52.427428 3216 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcabb0f147cf4254a6bd073c4802c00c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44769 } }
I20260926 06:52:52.427783 3218 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcabb0f147cf4254a6bd073c4802c00c. Latest consensus state: current_term: 1 leader_uuid: "bcabb0f147cf4254a6bd073c4802c00c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcabb0f147cf4254a6bd073c4802c00c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44769 } } }
I20260926 06:52:52.427778 3217 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcabb0f147cf4254a6bd073c4802c00c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcabb0f147cf4254a6bd073c4802c00c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44769 } } }
I20260926 06:52:52.427909 3218 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:52.427929 3217 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:52.428179 3225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:52:52.428746 3225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:52:52.428896 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 3145
I20260926 06:52:52.429014 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:52:52.429199 767 external_mini_cluster.cc:1428] Setting key fed026c4fd3b9cd5ace2a56d6bf00b0b
I20260926 06:52:52.430424 3225 catalog_manager.cc:1347] Generated new cluster ID: cf8c209736f6485e8dc305d3df8b0fd3
I20260926 06:52:52.430482 3225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:52:52 dist-test-slave-ws74 krb5kdc[3132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:52.440636 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:52.429745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56460 (local address 127.0.191.254:44769)
0926 06:52:52.429984 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:52.429992 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:52.430024 (+ 32us) server_negotiation.cc:407] Connection header received
0926 06:52:52.430138 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:52.430145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:52.430201 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:52.430346 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:52.431119 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:52.432528 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:52.433575 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:52.434236 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:52.434356 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:52.437102 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:52.437106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:52.437106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:52.437119 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:52.439052 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:52.439452 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:52.439455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:52.439456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:52.439555 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:52.439765 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:52.439766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:52.439766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:52.440043 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:52.440177 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:52.440296 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 06:52:52.440421 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":66,"threads_started":1}
I20260926 06:52:52.456835 3225 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:52:52.457365 3225 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:52:52.462697 3225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcabb0f147cf4254a6bd073c4802c00c: Generated new TSK 0
I20260926 06:52:52.462874 3225 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:52:52.495839 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:44769
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:52.580945 3239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:52.581173 3239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:52.581210 3239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:52.582369 3239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:52.582433 3239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:52.582479 3239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:52:52.583945 3239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:44769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:52.584199 3239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:52.584424 3239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:52:52.589118 3239 server_base.cc:1034] running on GCE node
W20260926 06:52:52.589108 3244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:52.589093 3245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:52.589102 3247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:52.589537 3239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:52.589808 3239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:52.590965 3239 hybrid_clock.cc:648] HybridClock initialized: now 1790405572590945 us; error 51 us; skew 500 ppm
Sep 26 06:52:52 dist-test-slave-ws74 krb5kdc[3132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405572, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:52.593230 3239 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:52:52.593730 3239 webserver.cc:460] Webserver started at http://127.0.191.193:46859/ using document root <none> and password file <none>
I20260926 06:52:52.593947 3239 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:52.594012 3239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:52.594177 3239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:52.595152 3239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "a603f82e426d459385e63ae010f69fc2"
format_stamp: "Formatted at 2026-09-26 06:52:52 on dist-test-slave-ws74"
server_key: "f3f069d74bc28279e258835f131692cd"
server_key_iv: "8465eb7f1105560f780d268186f327d9"
server_key_version: "encryptionkey@0"
I20260926 06:52:52.595498 3239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "a603f82e426d459385e63ae010f69fc2"
format_stamp: "Formatted at 2026-09-26 06:52:52 on dist-test-slave-ws74"
server_key: "f3f069d74bc28279e258835f131692cd"
server_key_iv: "8465eb7f1105560f780d268186f327d9"
server_key_version: "encryptionkey@0"
I20260926 06:52:52.596771 3239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:52:52.597470 3254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:52.597630 3239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:52:52.597746 3239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "a603f82e426d459385e63ae010f69fc2"
format_stamp: "Formatted at 2026-09-26 06:52:52 on dist-test-slave-ws74"
server_key: "f3f069d74bc28279e258835f131692cd"
server_key_iv: "8465eb7f1105560f780d268186f327d9"
server_key_version: "encryptionkey@0"
I20260926 06:52:52.597831 3239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:52.624578 3239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:52.627009 3239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:52.627197 3239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:52.627450 3239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:52:52.627784 3239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:52.627856 3239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:52.627895 3239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:52.627923 3239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:52.633844 3239 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:41875
I20260926 06:52:52.633893 3367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:41875 every 8 connection(s)
I20260926 06:52:52.634183 3239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:52:52 dist-test-slave-ws74 krb5kdc[3132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405572, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:52.640671 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 3239
I20260926 06:52:52.640772 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:52:52.640951 767 external_mini_cluster.cc:1428] Setting key d9da43fd61e8a853c872a975393cb8e7
I20260926 06:52:52.643289 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:52.634831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:33645 (local address 127.0.191.254:44769)
0926 06:52:52.634891 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:52.634894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:52.635071 (+ 177us) server_negotiation.cc:407] Connection header received
0926 06:52:52.635995 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:52.635996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:52.636008 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:52.636072 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:52.636655 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:52.637160 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:52.638169 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:52.638371 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:52.638424 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:52.640349 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:52.640350 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:52.640351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:52.640357 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:52.641927 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:52.642332 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:52.642333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:52.642333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:52.642389 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:52.642639 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:52.642640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:52.642641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:52.642765 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:52.642835 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:52.643118 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 06:52:52.643200 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:52:52.643484 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:52.634785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:44769 (local address 127.0.191.193:33645)
0926 06:52:52.634946 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:52.634986 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:52.635904 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:52.636148 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:52.636152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:52.636396 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:52.636581 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:52.636585 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:52.637235 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:52.637235 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:52.638096 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:52.638099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:52.638442 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:52.638442 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:52.638539 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:52.638980 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:52.638987 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:52.640265 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:52.642007 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:52.642014 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:52.642016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:52.642242 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:52.642469 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:52.642469 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:52.642470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:52.642573 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:52.642837 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:52.642843 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:52.642974 (+ 131us) client_negotiation.cc:769] Sending connection context
0926 06:52:52.643096 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 06:52:52.643253 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260926 06:52:52.643970 3368 heartbeater.cc:347] Connected to a master server at 127.0.191.254:44769
I20260926 06:52:52.644052 3368 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:52.644245 3368 heartbeater.cc:511] Master 127.0.191.254:44769 requested a full tablet report, sending...
I20260926 06:52:52.644763 3177 ts_manager.cc:194] Registered new tserver with Master: a603f82e426d459385e63ae010f69fc2 (127.0.191.193:41875)
I20260926 06:52:52.645351 3177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:33645
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:52:52.663828 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:44769
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:52.743130 3373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:52.743319 3373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:52.743341 3373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:52.744405 3373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:52.744439 3373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:52.744465 3373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:52:52.746127 3373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:44769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:52.746309 3373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:52.746470 3373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:52.751188 3381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:52.751348 3373 server_base.cc:1034] running on GCE node
W20260926 06:52:52.751209 3378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:52.751246 3379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:52.751722 3373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:52.751914 3373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:52.753046 3373 hybrid_clock.cc:648] HybridClock initialized: now 1790405572753037 us; error 46 us; skew 500 ppm
Sep 26 06:52:52 dist-test-slave-ws74 krb5kdc[3132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405572, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:52.755853 3373 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:52:52.756249 3373 webserver.cc:460] Webserver started at http://127.0.191.194:45519/ using document root <none> and password file <none>
I20260926 06:52:52.756480 3373 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:52.756547 3373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:52.756697 3373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:52.757834 3373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "68405ee8d3f74a0a80df94950238dc02"
format_stamp: "Formatted at 2026-09-26 06:52:52 on dist-test-slave-ws74"
server_key: "ead15712975b9261374a32c34ca07aee"
server_key_iv: "a6d11a57b196aa66a6deea7987b4330c"
server_key_version: "encryptionkey@0"
I20260926 06:52:52.758191 3373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "68405ee8d3f74a0a80df94950238dc02"
format_stamp: "Formatted at 2026-09-26 06:52:52 on dist-test-slave-ws74"
server_key: "ead15712975b9261374a32c34ca07aee"
server_key_iv: "a6d11a57b196aa66a6deea7987b4330c"
server_key_version: "encryptionkey@0"
I20260926 06:52:52.759481 3373 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 06:52:52.760169 3388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:52.760337 3373 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:52.760432 3373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "68405ee8d3f74a0a80df94950238dc02"
format_stamp: "Formatted at 2026-09-26 06:52:52 on dist-test-slave-ws74"
server_key: "ead15712975b9261374a32c34ca07aee"
server_key_iv: "a6d11a57b196aa66a6deea7987b4330c"
server_key_version: "encryptionkey@0"
I20260926 06:52:52.760516 3373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:52.780094 3373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:52.782593 3373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:52.782750 3373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:52.782982 3373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:52:52.783335 3373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:52.783391 3373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:52.783432 3373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:52.783457 3373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:52.789580 3373 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:33643
I20260926 06:52:52.789664 3501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:33643 every 8 connection(s)
I20260926 06:52:52.790004 3373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:52:52 dist-test-slave-ws74 krb5kdc[3132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405572, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:52.798306 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 3373
I20260926 06:52:52.798406 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:52:52.798619 767 external_mini_cluster.cc:1428] Setting key c0fb7d38bd71b84b1d6018e9668a50c4
I20260926 06:52:52.800701 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:52.790750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:44201 (local address 127.0.191.254:44769)
0926 06:52:52.790813 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:52.790816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:52.791015 (+ 199us) server_negotiation.cc:407] Connection header received
0926 06:52:52.792045 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:52.792046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:52.792055 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:52.792105 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:52.792701 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:52.793181 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:52.794235 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:52.794411 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:52.794468 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:52.797223 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:52.797224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:52.797225 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:52.797232 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:52.798906 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:52.799641 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:52.799642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:52.799643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:52.799705 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:52.799989 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:52.799990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:52.799990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:52.800140 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:52.800209 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:52.800461 (+ 252us) server_negotiation.cc:299] Negotiation successful
0926 06:52:52.800556 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 06:52:52.800968 3504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:52.790735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:44769 (local address 127.0.191.194:44201)
0926 06:52:52.790891 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:52.790930 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:52.791955 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:52.792166 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:52.792169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:52.792406 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:52.792627 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:52.792632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:52.793271 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:52.793272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:52.794163 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:52.794166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:52.794483 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:52.794484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:52.794580 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:52.795110 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:52.795117 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:52.797109 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:52.799222 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:52.799228 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:52.799230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:52.799555 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:52.799773 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:52.799774 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:52.799774 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:52.799914 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:52.800197 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:52.800201 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:52.800317 (+ 116us) client_negotiation.cc:769] Sending connection context
0926 06:52:52.800402 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 06:52:52.800606 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260926 06:52:52.801592 3502 heartbeater.cc:347] Connected to a master server at 127.0.191.254:44769
I20260926 06:52:52.801707 3502 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:52.801919 3502 heartbeater.cc:511] Master 127.0.191.254:44769 requested a full tablet report, sending...
I20260926 06:52:52.802400 3177 ts_manager.cc:194] Registered new tserver with Master: 68405ee8d3f74a0a80df94950238dc02 (127.0.191.194:33643)
I20260926 06:52:52.802945 3177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:44201
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
I20260926 06:52:53.646296 3368 heartbeater.cc:503] Master 127.0.191.254:44769 was elected leader, sending a full tablet report...
I20260926 06:52:53.803797 3502 heartbeater.cc:503] Master 127.0.191.254:44769 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
W20260926 06:52:53.812369 767 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.0.191.195: real 1.013s user 0.000s sys 0.003s
I20260926 06:52:53.812722 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:44769
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:53.891782 3507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:53.891983 3507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:53.892005 3507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:53.893368 3507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:53.893456 3507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:53.893500 3507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:52:53.895927 3507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:44769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:53.896212 3507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:53.896471 3507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:53.902355 3512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:53.902372 3515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:53.902355 3513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:53.902535 3507 server_base.cc:1034] running on GCE node
I20260926 06:52:53.902936 3507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:53.903254 3507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:53.904428 3507 hybrid_clock.cc:648] HybridClock initialized: now 1790405573904413 us; error 31 us; skew 500 ppm
Sep 26 06:52:53 dist-test-slave-ws74 krb5kdc[3132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405573, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:53.907258 3507 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:52:53.907682 3507 webserver.cc:460] Webserver started at http://127.0.191.195:34733/ using document root <none> and password file <none>
I20260926 06:52:53.907958 3507 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:53.908022 3507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:53.908155 3507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:53.909350 3507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "3ab80cddacd54f45aa32e465ab5077e0"
format_stamp: "Formatted at 2026-09-26 06:52:53 on dist-test-slave-ws74"
server_key: "1ba73b4f92ee16ea87a967392d6f6a16"
server_key_iv: "f67a46f83f2d40b9cfb551839c88c145"
server_key_version: "encryptionkey@0"
I20260926 06:52:53.909806 3507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "3ab80cddacd54f45aa32e465ab5077e0"
format_stamp: "Formatted at 2026-09-26 06:52:53 on dist-test-slave-ws74"
server_key: "1ba73b4f92ee16ea87a967392d6f6a16"
server_key_iv: "f67a46f83f2d40b9cfb551839c88c145"
server_key_version: "encryptionkey@0"
I20260926 06:52:53.913762 3507 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260926 06:52:53.914525 3522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:53.914692 3507 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:52:53.914793 3507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "3ab80cddacd54f45aa32e465ab5077e0"
format_stamp: "Formatted at 2026-09-26 06:52:53 on dist-test-slave-ws74"
server_key: "1ba73b4f92ee16ea87a967392d6f6a16"
server_key_iv: "f67a46f83f2d40b9cfb551839c88c145"
server_key_version: "encryptionkey@0"
I20260926 06:52:53.914875 3507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:53.941658 3507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:53.943490 3507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:53.943647 3507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:53.943919 3507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:52:53.944226 3507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:53.944278 3507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:53.944314 3507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:53.944340 3507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:53.950327 3507 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:37867
I20260926 06:52:53.950381 3635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:37867 every 8 connection(s)
I20260926 06:52:53.950721 3507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:52:53 dist-test-slave-ws74 krb5kdc[3132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405573, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:53.957506 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 3507
I20260926 06:52:53.957602 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:52:53.957849 767 external_mini_cluster.cc:1428] Setting key 318d1165b8c43cc0ad834d130745403c
I20260926 06:52:53.960239 3639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:53.951414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:38663 (local address 127.0.191.254:44769)
0926 06:52:53.952676 (+ 1262us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:53.952678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:53.952685 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:52:53.952708 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:53.952709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:53.952718 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:53.952768 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:53.953418 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:53.954019 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:53.954959 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:53.955142 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:53.955192 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:53.957105 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:53.957106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:53.957107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:53.957112 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:53.958889 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:53.959283 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:53.959284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:53.959285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:53.959347 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:53.959606 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:53.959607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:53.959607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:53.959722 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:53.959795 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:53.960057 (+ 262us) server_negotiation.cc:299] Negotiation successful
0926 06:52:53.960134 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1227,"thread_start_us":74,"threads_started":1}
I20260926 06:52:53.960384 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:53.951384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:44769 (local address 127.0.191.195:38663)
0926 06:52:53.951565 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:53.951615 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:53.952517 (+ 902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:53.952868 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:53.952871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:53.953109 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:53.953293 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:53.953297 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:53.954101 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:53.954102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:53.954887 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:53.954888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:53.955210 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:53.955210 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:53.955301 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:53.955723 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:53.955730 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:53.957027 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:53.958976 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:53.958982 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:53.958984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:53.959196 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:53.959427 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:53.959428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:53.959428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:53.959533 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:53.959787 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:53.959793 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:53.959926 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 06:52:53.960026 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 06:52:53.960169 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260926 06:52:53.960785 3636 heartbeater.cc:347] Connected to a master server at 127.0.191.254:44769
I20260926 06:52:53.960867 3636 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:53.960995 3636 heartbeater.cc:511] Master 127.0.191.254:44769 requested a full tablet report, sending...
I20260926 06:52:53.961340 3177 ts_manager.cc:194] Registered new tserver with Master: 3ab80cddacd54f45aa32e465ab5077e0 (127.0.191.195:37867)
I20260926 06:52:53.961711 3177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:38663
I20260926 06:52:53.969416 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 06:52:53.979647 3639 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:52:53.978914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56470 (local address 127.0.191.254:44769)
0926 06:52:53.978976 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:53.978978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:53.979044 (+ 66us) server_negotiation.cc:407] Connection header received
0926 06:52:53.979092 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:53.979092 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:53.979101 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:53.979152 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:53.979600 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56470: BlockingRecv error: recv got EOF from 127.0.0.1:56470 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 06:52:53.980530 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 3239
I20260926 06:52:53.985421 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 3373
I20260926 06:52:53.989552 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 3507
I20260926 06:52:53.993597 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 3145
2026-09-26T06:52:53Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2830 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:52:55 dist-test-slave-ws74 krb5kdc[3649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:52:55 dist-test-slave-ws74 krb5kdc[3649](info): set up 2 sockets
Sep 26 06:52:55 dist-test-slave-ws74 krb5kdc[3649](info): commencing operation
krb5kdc: starting...
W20260926 06:52:56.952538 767 mini_kdc.cc:121] Time spent starting KDC: real 2.938s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260926 06:52:57.159789 767 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.207s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:52:57 dist-test-slave-ws74 krb5kdc[3649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:52:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:52:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:52:57.270429 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:42811
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:42811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:57.349453 3662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:57.349675 3662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:57.349709 3662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:57.350835 3662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:52:57.350905 3662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:57.350930 3662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:57.350956 3662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:52:57.350983 3662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:52:57.352423 3662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:42811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:42811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:57.352685 3662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:57.352897 3662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:57.357579 3668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:57.357595 3670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:57.357589 3667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:57.357887 3662 server_base.cc:1034] running on GCE node
I20260926 06:52:57.358081 3662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:57.358314 3662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:57.359449 3662 hybrid_clock.cc:648] HybridClock initialized: now 1790405577359423 us; error 28 us; skew 500 ppm
Sep 26 06:52:57 dist-test-slave-ws74 krb5kdc[3649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405577, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:57.362013 3662 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:52:57.362481 3662 webserver.cc:460] Webserver started at http://127.0.191.254:39797/ using document root <none> and password file <none>
I20260926 06:52:57.362656 3662 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:57.362696 3662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:57.362787 3662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:57.363767 3662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "731ab24d4d55458ba1b53f6cae120f1c"
format_stamp: "Formatted at 2026-09-26 06:52:57 on dist-test-slave-ws74"
server_key: "616b5dd05d5c3ba3bbc0e123f4b6b873"
server_key_iv: "7c24b4488861598a6b14b385bed3b8f4"
server_key_version: "encryptionkey@0"
I20260926 06:52:57.364063 3662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "731ab24d4d55458ba1b53f6cae120f1c"
format_stamp: "Formatted at 2026-09-26 06:52:57 on dist-test-slave-ws74"
server_key: "616b5dd05d5c3ba3bbc0e123f4b6b873"
server_key_iv: "7c24b4488861598a6b14b385bed3b8f4"
server_key_version: "encryptionkey@0"
I20260926 06:52:57.365324 3662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:52:57.366076 3677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:57.366240 3662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:52:57.366346 3662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "731ab24d4d55458ba1b53f6cae120f1c"
format_stamp: "Formatted at 2026-09-26 06:52:57 on dist-test-slave-ws74"
server_key: "616b5dd05d5c3ba3bbc0e123f4b6b873"
server_key_iv: "7c24b4488861598a6b14b385bed3b8f4"
server_key_version: "encryptionkey@0"
I20260926 06:52:57.366400 3662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:57.389406 3662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:57.391170 3662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:57.391278 3662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:57.395095 3662 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:42811
I20260926 06:52:57.395148 3729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:42811 every 8 connection(s)
I20260926 06:52:57.395630 3662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:52:57.396301 3730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:57.398579 3730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c: Bootstrap starting.
I20260926 06:52:57.399194 3730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:57.399501 3730 log.cc:824] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:57.400159 3730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c: No bootstrap required, opened a new log
I20260926 06:52:57.401727 3730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731ab24d4d55458ba1b53f6cae120f1c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42811 } }
I20260926 06:52:57.401894 3730 raft_consensus.cc:374] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:57.401942 3730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 731ab24d4d55458ba1b53f6cae120f1c, State: Initialized, Role: FOLLOWER
I20260926 06:52:57.402125 3730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731ab24d4d55458ba1b53f6cae120f1c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42811 } }
I20260926 06:52:57.402226 3730 raft_consensus.cc:388] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:52:57.402266 3730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:52:57.402318 3730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:57.403260 3730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731ab24d4d55458ba1b53f6cae120f1c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42811 } }
I20260926 06:52:57.403380 3730 leader_election.cc:302] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 731ab24d4d55458ba1b53f6cae120f1c; no voters:
I20260926 06:52:57.403559 3730 leader_election.cc:288] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:52:57.403602 3733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:57.403787 3733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [term 1 LEADER]: Becoming Leader. State: Replica: 731ab24d4d55458ba1b53f6cae120f1c, State: Running, Role: LEADER
I20260926 06:52:57.403815 3730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:52:57.403872 3733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731ab24d4d55458ba1b53f6cae120f1c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42811 } }
I20260926 06:52:57.404230 3734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "731ab24d4d55458ba1b53f6cae120f1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731ab24d4d55458ba1b53f6cae120f1c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42811 } } }
I20260926 06:52:57.404235 3735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 731ab24d4d55458ba1b53f6cae120f1c. Latest consensus state: current_term: 1 leader_uuid: "731ab24d4d55458ba1b53f6cae120f1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731ab24d4d55458ba1b53f6cae120f1c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42811 } } }
I20260926 06:52:57.404371 3735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:57.404706 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 3662
I20260926 06:52:57.404805 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:52:57.404884 3742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:52:57.404364 3734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:57.404983 767 external_mini_cluster.cc:1428] Setting key 4b4177fa7776118991eacb09de9c9259
I20260926 06:52:57.405418 3742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:52:57.407027 3742 catalog_manager.cc:1347] Generated new cluster ID: ead0672eda4a4acfa017fa2615ff7e25
I20260926 06:52:57.407079 3742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:52:57 dist-test-slave-ws74 krb5kdc[3649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:57.415114 3752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:57.406025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47912 (local address 127.0.191.254:42811)
0926 06:52:57.406239 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:57.406260 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:57.406291 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:52:57.406402 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:57.406409 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:57.406482 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:57.406624 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:57.407317 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:57.408230 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:57.408989 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:57.409494 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:57.409617 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:57.411892 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:57.411896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:57.411896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:57.411907 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:57.413612 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:57.413971 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:57.413973 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:57.413974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:57.414066 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:57.414278 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:57.414279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:57.414280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:57.414504 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:57.414668 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:57.414787 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 06:52:57.414922 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":73,"threads_started":1}
I20260926 06:52:57.415973 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:52:57.434736 3752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:57.430647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47922 (local address 127.0.191.254:42811)
0926 06:52:57.430690 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:57.430692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:57.430825 (+ 133us) server_negotiation.cc:407] Connection header received
0926 06:52:57.430890 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:57.430891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:57.430901 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:57.430963 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:57.431416 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:57.432117 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:57.433153 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:57.433503 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:57.433561 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:57.434249 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:57.434251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:57.434251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:52:57.434483 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:57.434499 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:52:57.434505 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:57.434557 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:57.434635 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 06:52:57.434657 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:52:57.439570 3742 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:52:57.440068 3742 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:52:57.446520 3742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 731ab24d4d55458ba1b53f6cae120f1c: Generated new TSK 0
I20260926 06:52:57.446694 3742 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:52:57.450732 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 3662
2026-09-26T06:52:57Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (3454 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:52:57 dist-test-slave-ws74 krb5kdc[3763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:52:57 dist-test-slave-ws74 krb5kdc[3763](info): set up 2 sockets
Sep 26 06:52:57 dist-test-slave-ws74 krb5kdc[3763](info): commencing operation
krb5kdc: starting...
W20260926 06:52:58.489259 767 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:52:58 dist-test-slave-ws74 krb5kdc[3763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:52:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:52:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:52:58.600410 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33183
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:33183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:58.678139 3776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:58.678377 3776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:58.678411 3776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:58.679538 3776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:52:58.679601 3776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:58.679625 3776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:58.679646 3776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:52:58.679667 3776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:52:58.681023 3776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:33183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:58.681286 3776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:58.681517 3776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:58.686748 3784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:58.686774 3781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:58.686767 3782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:58.686826 3776 server_base.cc:1034] running on GCE node
I20260926 06:52:58.687150 3776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:58.687376 3776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:58.688551 3776 hybrid_clock.cc:648] HybridClock initialized: now 1790405578688537 us; error 27 us; skew 500 ppm
Sep 26 06:52:58 dist-test-slave-ws74 krb5kdc[3763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405578, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:58.690881 3776 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:52:58.691265 3776 webserver.cc:460] Webserver started at http://127.0.191.254:45115/ using document root <none> and password file <none>
I20260926 06:52:58.691450 3776 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:58.691489 3776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:58.691571 3776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:58.692555 3776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "68054cca8f2347f1ac2dc59c71deb693"
format_stamp: "Formatted at 2026-09-26 06:52:58 on dist-test-slave-ws74"
server_key: "4406febfd63edb3fe4f828570e7a190f"
server_key_iv: "0d4ffc0314ccb2f26d22c6579c55d870"
server_key_version: "encryptionkey@0"
I20260926 06:52:58.692849 3776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "68054cca8f2347f1ac2dc59c71deb693"
format_stamp: "Formatted at 2026-09-26 06:52:58 on dist-test-slave-ws74"
server_key: "4406febfd63edb3fe4f828570e7a190f"
server_key_iv: "0d4ffc0314ccb2f26d22c6579c55d870"
server_key_version: "encryptionkey@0"
I20260926 06:52:58.694118 3776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:52:58.694783 3791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:58.694948 3776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:52:58.695022 3776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "68054cca8f2347f1ac2dc59c71deb693"
format_stamp: "Formatted at 2026-09-26 06:52:58 on dist-test-slave-ws74"
server_key: "4406febfd63edb3fe4f828570e7a190f"
server_key_iv: "0d4ffc0314ccb2f26d22c6579c55d870"
server_key_version: "encryptionkey@0"
I20260926 06:52:58.695073 3776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:58.720526 3776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:58.722397 3776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:58.722551 3776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:58.726289 3776 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:33183
I20260926 06:52:58.726337 3843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:33183 every 8 connection(s)
I20260926 06:52:58.726680 3776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:52:58.727641 3844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:52:58.729879 3844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693: Bootstrap starting.
I20260926 06:52:58.730386 3844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693: Neither blocks nor log segments found. Creating new log.
I20260926 06:52:58.730607 3844 log.cc:824] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693: Log is configured to *not* fsync() on all Append() calls
I20260926 06:52:58.731212 3844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693: No bootstrap required, opened a new log
I20260926 06:52:58.732689 3844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68054cca8f2347f1ac2dc59c71deb693" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33183 } }
I20260926 06:52:58.732836 3844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:52:58.732857 3844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68054cca8f2347f1ac2dc59c71deb693, State: Initialized, Role: FOLLOWER
I20260926 06:52:58.732952 3844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68054cca8f2347f1ac2dc59c71deb693" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33183 } }
I20260926 06:52:58.732999 3844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:52:58.733019 3844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:52:58.733049 3844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:52:58.733834 3844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68054cca8f2347f1ac2dc59c71deb693" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33183 } }
I20260926 06:52:58.733917 3844 leader_election.cc:302] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68054cca8f2347f1ac2dc59c71deb693; no voters:
I20260926 06:52:58.734064 3844 leader_election.cc:288] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:52:58.734094 3847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:52:58.734220 3847 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [term 1 LEADER]: Becoming Leader. State: Replica: 68054cca8f2347f1ac2dc59c71deb693, State: Running, Role: LEADER
I20260926 06:52:58.734308 3844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:52:58.734364 3847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68054cca8f2347f1ac2dc59c71deb693" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33183 } }
I20260926 06:52:58.734750 3849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68054cca8f2347f1ac2dc59c71deb693. Latest consensus state: current_term: 1 leader_uuid: "68054cca8f2347f1ac2dc59c71deb693" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68054cca8f2347f1ac2dc59c71deb693" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33183 } } }
I20260926 06:52:58.734844 3849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:58.734890 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 3776
I20260926 06:52:58.734994 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:52:58.735168 767 external_mini_cluster.cc:1428] Setting key 6e2cd495fc14f115ced2027d24503325
I20260926 06:52:58.735138 3856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:52:58.735531 3848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68054cca8f2347f1ac2dc59c71deb693" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68054cca8f2347f1ac2dc59c71deb693" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33183 } } }
I20260926 06:52:58.735608 3848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693 [sys.catalog]: This master's current role is: LEADER
I20260926 06:52:58.735661 3856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:52:58.737321 3856 catalog_manager.cc:1347] Generated new cluster ID: 89c1a56c669646a9bba4e9e7fd00f497
I20260926 06:52:58.737393 3856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:52:58 dist-test-slave-ws74 krb5kdc[3763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:58.744956 3865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:58.736115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42822 (local address 127.0.191.254:33183)
0926 06:52:58.736258 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:58.736267 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:58.736299 (+ 32us) server_negotiation.cc:407] Connection header received
0926 06:52:58.736428 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:58.736436 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:58.736493 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:58.737400 (+ 907us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:58.737493 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:58.738356 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:58.739136 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:58.739544 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:58.739668 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:58.741851 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:58.741855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:58.741856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:58.741872 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:58.743470 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:58.743833 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:58.743836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:58.743837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:58.743933 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:58.744139 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:58.744140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:58.744140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:58.744368 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:58.744520 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:58.744621 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 06:52:58.744768 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":69,"threads_started":1}
I20260926 06:52:58.760164 3856 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:52:58.760849 3856 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:52:58.770263 3856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68054cca8f2347f1ac2dc59c71deb693: Generated new TSK 0
I20260926 06:52:58.770449 3856 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:52:58.799748 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33183
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:58.898126 3870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:58.898319 3870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:58.898340 3870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:58.899564 3870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:58.899605 3870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:58.899657 3870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:52:58.901024 3870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:58.901274 3870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:58.901525 3870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:58.906283 3875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:58.906343 3870 server_base.cc:1034] running on GCE node
W20260926 06:52:58.906309 3878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:58.906284 3876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:58.906737 3870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:58.906922 3870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:58.908102 3870 hybrid_clock.cc:648] HybridClock initialized: now 1790405578908094 us; error 34 us; skew 500 ppm
Sep 26 06:52:58 dist-test-slave-ws74 krb5kdc[3763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405578, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:58.910553 3870 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:52:58.910949 3870 webserver.cc:460] Webserver started at http://127.0.191.193:41057/ using document root <none> and password file <none>
I20260926 06:52:58.911162 3870 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:58.911206 3870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:58.911353 3870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:58.912356 3870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "46b2ef303dec4800895d13a0731f263c"
format_stamp: "Formatted at 2026-09-26 06:52:58 on dist-test-slave-ws74"
server_key: "79de1737e3a726b663c3a4d6ed440199"
server_key_iv: "f17e3c5eebb9f8fb840c95f8beeae098"
server_key_version: "encryptionkey@0"
I20260926 06:52:58.912708 3870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "46b2ef303dec4800895d13a0731f263c"
format_stamp: "Formatted at 2026-09-26 06:52:58 on dist-test-slave-ws74"
server_key: "79de1737e3a726b663c3a4d6ed440199"
server_key_iv: "f17e3c5eebb9f8fb840c95f8beeae098"
server_key_version: "encryptionkey@0"
I20260926 06:52:58.914054 3870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:52:58.914785 3885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:58.914966 3870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:52:58.915052 3870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "46b2ef303dec4800895d13a0731f263c"
format_stamp: "Formatted at 2026-09-26 06:52:58 on dist-test-slave-ws74"
server_key: "79de1737e3a726b663c3a4d6ed440199"
server_key_iv: "f17e3c5eebb9f8fb840c95f8beeae098"
server_key_version: "encryptionkey@0"
I20260926 06:52:58.915122 3870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:58.938616 3870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:58.940371 3870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:58.940522 3870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:58.940747 3870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:52:58.941032 3870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:58.941064 3870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:58.941087 3870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:58.941130 3870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:58.946712 3870 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:42205
I20260926 06:52:58.946906 3998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:42205 every 8 connection(s)
I20260926 06:52:58.947075 3870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:52:58 dist-test-slave-ws74 krb5kdc[3763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405578, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:58.955106 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 3870
I20260926 06:52:58.955209 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:52:58.955426 767 external_mini_cluster.cc:1428] Setting key 53f43d1dc98d0c9c49e98efcc76e2bb3
I20260926 06:52:58.956279 3865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:58.947822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:34567 (local address 127.0.191.254:33183)
0926 06:52:58.947866 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:58.947867 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:58.948063 (+ 196us) server_negotiation.cc:407] Connection header received
0926 06:52:58.948533 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:58.948534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:58.948545 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:58.948595 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:58.949519 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:58.950008 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:58.950911 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:58.951102 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:58.951150 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:58.953245 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:58.953246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:58.953247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:58.953253 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:58.954695 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:58.955215 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:58.955216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:58.955217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:58.955292 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:58.955644 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:58.955645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:58.955645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:58.955808 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:58.955871 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:58.956127 (+ 256us) server_negotiation.cc:299] Negotiation successful
0926 06:52:58.956215 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:52:58.956552 4001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:58.947817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33183 (local address 127.0.191.193:34567)
0926 06:52:58.947949 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:58.947982 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:58.948454 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:58.948648 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:58.948651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:58.949200 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:58.949446 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:58.949451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:58.950080 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:58.950081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:58.950843 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:58.950845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:58.951181 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:58.951182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:58.951275 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:58.951797 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:58.951805 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:58.953162 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:58.954761 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:58.954765 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:58.954767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:58.955110 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:58.955376 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:58.955377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:58.955377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:58.955503 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:58.955896 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:58.955909 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:58.956019 (+ 110us) client_negotiation.cc:769] Sending connection context
0926 06:52:58.956100 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 06:52:58.956281 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
I20260926 06:52:58.956981 3999 heartbeater.cc:347] Connected to a master server at 127.0.191.254:33183
I20260926 06:52:58.957070 3999 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:58.957266 3999 heartbeater.cc:511] Master 127.0.191.254:33183 requested a full tablet report, sending...
I20260926 06:52:58.957762 3808 ts_manager.cc:194] Registered new tserver with Master: 46b2ef303dec4800895d13a0731f263c (127.0.191.193:42205)
I20260926 06:52:58.958374 3808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:34567
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:52:58.979940 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33183
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:59.057536 4004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:59.057770 4004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:59.057806 4004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:59.058929 4004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:59.059000 4004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:59.059049 4004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:52:59.060465 4004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:59.060726 4004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:59.060954 4004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:59.065768 4009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:59.065760 4012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:59.065915 4004 server_base.cc:1034] running on GCE node
W20260926 06:52:59.065732 4010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:59.066134 4004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:59.066376 4004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:59.067533 4004 hybrid_clock.cc:648] HybridClock initialized: now 1790405579067505 us; error 38 us; skew 500 ppm
Sep 26 06:52:59 dist-test-slave-ws74 krb5kdc[3763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405579, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:59.070770 4004 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:52:59.071292 4004 webserver.cc:460] Webserver started at http://127.0.191.194:39455/ using document root <none> and password file <none>
I20260926 06:52:59.071545 4004 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:59.071602 4004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:59.071729 4004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:59.072997 4004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "d18986d075074935b259e47002474b12"
format_stamp: "Formatted at 2026-09-26 06:52:59 on dist-test-slave-ws74"
server_key: "14d80bf068e8d22dcedd638b760c7784"
server_key_iv: "496f28bd64b92daa5b3cbb32f0758be8"
server_key_version: "encryptionkey@0"
I20260926 06:52:59.073433 4004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "d18986d075074935b259e47002474b12"
format_stamp: "Formatted at 2026-09-26 06:52:59 on dist-test-slave-ws74"
server_key: "14d80bf068e8d22dcedd638b760c7784"
server_key_iv: "496f28bd64b92daa5b3cbb32f0758be8"
server_key_version: "encryptionkey@0"
I20260926 06:52:59.075006 4004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:52:59.075670 4019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:59.075847 4004 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:52:59.075940 4004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "d18986d075074935b259e47002474b12"
format_stamp: "Formatted at 2026-09-26 06:52:59 on dist-test-slave-ws74"
server_key: "14d80bf068e8d22dcedd638b760c7784"
server_key_iv: "496f28bd64b92daa5b3cbb32f0758be8"
server_key_version: "encryptionkey@0"
I20260926 06:52:59.076025 4004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:59.092021 4004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:59.094527 4004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:59.094681 4004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:59.094890 4004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:52:59.095186 4004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:59.095242 4004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:59.095283 4004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:59.095316 4004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:59.101341 4004 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:39597
I20260926 06:52:59.101425 4132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:39597 every 8 connection(s)
I20260926 06:52:59.101751 4004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:52:59.104030 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 4004
I20260926 06:52:59.104151 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:52:59.104347 767 external_mini_cluster.cc:1428] Setting key 3ef221da42c2f807e4f749a15c265dae
Sep 26 06:52:59 dist-test-slave-ws74 krb5kdc[3763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405579, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:59.112142 4135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:59.102430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33183 (local address 127.0.191.194:54413)
0926 06:52:59.102583 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:59.102624 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:59.103162 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:59.103415 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:59.103419 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:59.104002 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:59.104252 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:59.104262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:59.105093 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:59.105095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:59.106467 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:59.106470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:59.106972 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:59.106973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:59.107070 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:59.107593 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:59.107608 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:59.108985 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:59.110858 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:59.110863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:59.110864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:59.111062 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:59.111274 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:59.111275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:59.111275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:59.111367 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:59.111649 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:59.111652 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:59.111799 (+ 147us) client_negotiation.cc:769] Sending connection context
0926 06:52:59.111881 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 06:52:59.111992 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260926 06:52:59.112149 3865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:59.102447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:54413 (local address 127.0.191.254:33183)
0926 06:52:59.102510 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:59.102512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:59.102712 (+ 200us) server_negotiation.cc:407] Connection header received
0926 06:52:59.103272 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:59.103273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:59.103286 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:59.103346 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:59.104368 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:59.104986 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:59.106563 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:59.106882 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:59.106964 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:59.109128 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:59.109130 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:59.109130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:59.109139 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:59.110775 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:59.111136 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:59.111137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:59.111138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:59.111201 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:59.111437 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:59.111438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:59.111438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:59.111579 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:59.111654 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:59.111930 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 06:52:59.112020 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
I20260926 06:52:59.112944 4133 heartbeater.cc:347] Connected to a master server at 127.0.191.254:33183
I20260926 06:52:59.113056 4133 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:59.113251 4133 heartbeater.cc:511] Master 127.0.191.254:33183 requested a full tablet report, sending...
I20260926 06:52:59.113629 3808 ts_manager.cc:194] Registered new tserver with Master: d18986d075074935b259e47002474b12 (127.0.191.194:39597)
I20260926 06:52:59.114049 3808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:54413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:52:59.128618 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33183
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:52:59.208892 4138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:52:59.209077 4138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:52:59.209100 4138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:52:59.210242 4138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:52:59.210280 4138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:52:59.210305 4138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:52:59.211609 4138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:52:59.211769 4138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:52:59.211943 4138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:52:59.216650 4146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:52:59.216698 4143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:59.216684 4138 server_base.cc:1034] running on GCE node
W20260926 06:52:59.216696 4144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:52:59.217139 4138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:52:59.217428 4138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:52:59.218570 4138 hybrid_clock.cc:648] HybridClock initialized: now 1790405579218555 us; error 32 us; skew 500 ppm
Sep 26 06:52:59 dist-test-slave-ws74 krb5kdc[3763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405579, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:52:59.220906 4138 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:52:59.221259 4138 webserver.cc:460] Webserver started at http://127.0.191.195:42563/ using document root <none> and password file <none>
I20260926 06:52:59.221493 4138 fs_manager.cc:359] Metadata directory not provided
I20260926 06:52:59.221536 4138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:52:59.221621 4138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:52:59.222659 4138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "7bc76f9b3a7148b3b2704c0fbdec1187"
format_stamp: "Formatted at 2026-09-26 06:52:59 on dist-test-slave-ws74"
server_key: "3ed02f4f270ba47245f28653edf4f18f"
server_key_iv: "c01de5f19d214196f3bb58a890021aa5"
server_key_version: "encryptionkey@0"
I20260926 06:52:59.222949 4138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "7bc76f9b3a7148b3b2704c0fbdec1187"
format_stamp: "Formatted at 2026-09-26 06:52:59 on dist-test-slave-ws74"
server_key: "3ed02f4f270ba47245f28653edf4f18f"
server_key_iv: "c01de5f19d214196f3bb58a890021aa5"
server_key_version: "encryptionkey@0"
I20260926 06:52:59.224275 4138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:52:59.224921 4153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:59.225067 4138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:52:59.225152 4138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "7bc76f9b3a7148b3b2704c0fbdec1187"
format_stamp: "Formatted at 2026-09-26 06:52:59 on dist-test-slave-ws74"
server_key: "3ed02f4f270ba47245f28653edf4f18f"
server_key_iv: "c01de5f19d214196f3bb58a890021aa5"
server_key_version: "encryptionkey@0"
I20260926 06:52:59.225203 4138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:52:59.237536 4138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:52:59.239161 4138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:52:59.239265 4138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:52:59.239423 4138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:52:59.239706 4138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:52:59.239744 4138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:59.239764 4138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:52:59.239812 4138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:52:59.245307 4138 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:33049
I20260926 06:52:59.245347 4266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:33049 every 8 connection(s)
I20260926 06:52:59.245685 4138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:52:59 dist-test-slave-ws74 krb5kdc[3763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405579, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:59.252982 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 4138
I20260926 06:52:59.253072 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:52:59.253259 767 external_mini_cluster.cc:1428] Setting key 14fa05650d218e586fd8ac79c7dedba5
I20260926 06:52:59.254412 3865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:59.246370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:37913 (local address 127.0.191.254:33183)
0926 06:52:59.246412 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:59.246414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:59.246647 (+ 233us) server_negotiation.cc:407] Connection header received
0926 06:52:59.247090 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:59.247091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:59.247101 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:59.247152 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:59.248012 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:59.248503 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:59.249487 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:59.249665 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:59.249716 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:59.251681 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:59.251682 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:59.251682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:59.251687 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:59.252841 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:59.253318 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:59.253319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:59.253320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:59.253402 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:59.253738 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:59.253739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:59.253739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:59.253891 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:59.253954 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:59.254290 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 06:52:59.254354 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:52:59.254581 4269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:59.246348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33183 (local address 127.0.191.195:37913)
0926 06:52:59.246491 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 06:52:59.246522 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 06:52:59.247011 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:52:59.247211 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:52:59.247214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:52:59.247758 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:52:59.247941 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:59.247945 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:59.248577 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:59.248577 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:59.249356 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:52:59.249359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:52:59.249731 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:52:59.249732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:52:59.249830 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:59.250325 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:52:59.250333 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:52:59.251606 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:52:59.252914 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:59.252921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:59.252923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:59.253233 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:59.253459 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:52:59.253460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:52:59.253460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:52:59.253617 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:52:59.253959 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:52:59.253966 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:52:59.254128 (+ 162us) client_negotiation.cc:769] Sending connection context
0926 06:52:59.254252 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 06:52:59.254396 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":83,"threads_started":1}
I20260926 06:52:59.255015 4267 heartbeater.cc:347] Connected to a master server at 127.0.191.254:33183
I20260926 06:52:59.255112 4267 heartbeater.cc:464] Registering TS with master...
I20260926 06:52:59.255277 4267 heartbeater.cc:511] Master 127.0.191.254:33183 requested a full tablet report, sending...
I20260926 06:52:59.255653 3808 ts_manager.cc:194] Registered new tserver with Master: 7bc76f9b3a7148b3b2704c0fbdec1187 (127.0.191.195:33049)
I20260926 06:52:59.256021 3808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:37913
I20260926 06:52:59.265008 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 06:52:59 dist-test-slave-ws74 krb5kdc[3763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405579, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 06:52:59 dist-test-slave-ws74 krb5kdc[3763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405579, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:52:59.286650 3865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:52:59.279729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.0.191.254:33183)
0926 06:52:59.279782 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 06:52:59.279784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:52:59.280009 (+ 225us) server_negotiation.cc:407] Connection header received
0926 06:52:59.280070 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:52:59.280071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:52:59.280084 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:52:59.280156 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:52:59.280873 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:59.281325 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:59.282046 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:52:59.282218 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:52:59.282267 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:52:59.284339 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:52:59.284340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:52:59.284341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:52:59.284348 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:52:59.285539 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:59.285859 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:59.285860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:59.285861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:59.285916 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:52:59.286124 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:52:59.286125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:52:59.286126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:52:59.286266 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:52:59.286368 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 06:52:59.286480 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 06:52:59.286551 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260926 06:52:59.286854 3808 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42828
I20260926 06:52:59.287245 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 3870
I20260926 06:52:59.291710 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 4004
I20260926 06:52:59.295810 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 4138
I20260926 06:52:59.299964 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 3776
2026-09-26T06:52:59Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1851 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:52:59 dist-test-slave-ws74 krb5kdc[4279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:52:59 dist-test-slave-ws74 krb5kdc[4279](info): set up 2 sockets
Sep 26 06:52:59 dist-test-slave-ws74 krb5kdc[4279](info): commencing operation
krb5kdc: starting...
W20260926 06:53:00.325331 767 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:53:00 dist-test-slave-ws74 krb5kdc[4279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:53:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:53:00.439666 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:42759
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:44303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:42759
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:00.520313 4292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:00.520514 4292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:00.520536 4292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:00.521775 4292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:00.521817 4292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:00.521832 4292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:00.521845 4292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:00.521857 4292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:00.523335 4292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:44303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:42759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:42759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:00.523576 4292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:00.523809 4292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:00.528529 4297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:00.528534 4300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:00.528715 4298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:00.528962 4292 server_base.cc:1034] running on GCE node
I20260926 06:53:00.529160 4292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:00.529431 4292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:00.530606 4292 hybrid_clock.cc:648] HybridClock initialized: now 1790405580530560 us; error 64 us; skew 500 ppm
Sep 26 06:53:00 dist-test-slave-ws74 krb5kdc[4279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405580, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:00.539340 4292 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:53:00.539764 4292 webserver.cc:460] Webserver started at http://127.0.191.254:44697/ using document root <none> and password file <none>
I20260926 06:53:00.539942 4292 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:00.539983 4292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:00.540071 4292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:00.541072 4292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "bd0a768ed9e54f27893c372841c93f0c"
format_stamp: "Formatted at 2026-09-26 06:53:00 on dist-test-slave-ws74"
server_key: "71eac728450cd77c783de9b70df0eb01"
server_key_iv: "ad111037ea9b4612577de0a005fed9dd"
server_key_version: "encryptionkey@0"
I20260926 06:53:00.541425 4292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "bd0a768ed9e54f27893c372841c93f0c"
format_stamp: "Formatted at 2026-09-26 06:53:00 on dist-test-slave-ws74"
server_key: "71eac728450cd77c783de9b70df0eb01"
server_key_iv: "ad111037ea9b4612577de0a005fed9dd"
server_key_version: "encryptionkey@0"
I20260926 06:53:00.542716 4292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:00.543538 4307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:00.543751 4292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:00.543833 4292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "bd0a768ed9e54f27893c372841c93f0c"
format_stamp: "Formatted at 2026-09-26 06:53:00 on dist-test-slave-ws74"
server_key: "71eac728450cd77c783de9b70df0eb01"
server_key_iv: "ad111037ea9b4612577de0a005fed9dd"
server_key_version: "encryptionkey@0"
I20260926 06:53:00.543917 4292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:00.562999 4292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:00.565348 4292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:00.565557 4292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:00.569504 4292 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:42759
I20260926 06:53:00.569561 4359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:42759 every 8 connection(s)
I20260926 06:53:00.569900 4292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:00.570775 4360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:00.572965 4360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c: Bootstrap starting.
I20260926 06:53:00.573611 4360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:00.573897 4360 log.cc:824] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:00.574262 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 4292
I20260926 06:53:00.574384 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:00.574573 767 external_mini_cluster.cc:1428] Setting key 5bc0ed026f26fd565217c39d27dac12b
I20260926 06:53:00.574637 4360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c: No bootstrap required, opened a new log
I20260926 06:53:00.576499 4360 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd0a768ed9e54f27893c372841c93f0c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42759 } }
I20260926 06:53:00.576640 4360 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:00.576700 4360 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd0a768ed9e54f27893c372841c93f0c, State: Initialized, Role: FOLLOWER
I20260926 06:53:00.576826 4360 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd0a768ed9e54f27893c372841c93f0c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42759 } }
I20260926 06:53:00.576885 4360 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:00.576906 4360 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:00.576967 4360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:00.577524 4360 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd0a768ed9e54f27893c372841c93f0c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42759 } }
I20260926 06:53:00.577607 4360 leader_election.cc:302] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd0a768ed9e54f27893c372841c93f0c; no voters:
I20260926 06:53:00.577802 4360 leader_election.cc:288] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:00.577849 4365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:00.577977 4365 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [term 1 LEADER]: Becoming Leader. State: Replica: bd0a768ed9e54f27893c372841c93f0c, State: Running, Role: LEADER
I20260926 06:53:00.578056 4360 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:00.578053 4365 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd0a768ed9e54f27893c372841c93f0c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42759 } }
I20260926 06:53:00.578447 4366 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd0a768ed9e54f27893c372841c93f0c. Latest consensus state: current_term: 1 leader_uuid: "bd0a768ed9e54f27893c372841c93f0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd0a768ed9e54f27893c372841c93f0c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42759 } } }
I20260926 06:53:00.578456 4367 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd0a768ed9e54f27893c372841c93f0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd0a768ed9e54f27893c372841c93f0c" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42759 } } }
I20260926 06:53:00.578534 4366 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:00.578542 4367 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:00.578789 4369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:00.579276 4369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:00.581565 4369 catalog_manager.cc:1347] Generated new cluster ID: 1a3547f065344e34b96a15b890153cb3
I20260926 06:53:00.581619 4369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:53:00 dist-test-slave-ws74 krb5kdc[4279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:00.586612 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:00.575065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39250 (local address 127.0.191.254:42759)
0926 06:53:00.575286 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:00.575298 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:00.575335 (+ 37us) server_negotiation.cc:407] Connection header received
0926 06:53:00.575879 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:00.575885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:00.575931 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:00.576076 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:00.576719 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:00.578236 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:00.580420 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:00.581010 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:00.581133 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:00.583436 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:00.583440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:00.583440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:00.583458 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:00.585054 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:00.585446 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:00.585449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:00.585449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:00.585532 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:00.585769 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:00.585770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:00.585770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:00.586050 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:00.586215 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:00.586327 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 06:53:00.586457 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":107,"threads_started":1}
I20260926 06:53:00.597895 4369 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:00.598449 4369 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:00.606736 4369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd0a768ed9e54f27893c372841c93f0c: Generated new TSK 0
I20260926 06:53:00.606920 4369 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:53:00.636544 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42759
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:44303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:00.719515 4386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:00.719702 4386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:00.719724 4386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:00.720834 4386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:00.720888 4386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:00.720914 4386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:53:00.722337 4386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:44303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:00.722508 4386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:00.722661 4386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:00.727437 4391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:00.727448 4392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:00.727451 4394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:00.727635 4386 server_base.cc:1034] running on GCE node
I20260926 06:53:00.727849 4386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:00.728071 4386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:00.729204 4386 hybrid_clock.cc:648] HybridClock initialized: now 1790405580729197 us; error 30 us; skew 500 ppm
Sep 26 06:53:00 dist-test-slave-ws74 krb5kdc[4279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405580, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:00.731577 4386 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:53:00.731973 4386 webserver.cc:460] Webserver started at http://127.0.191.193:38647/ using document root <none> and password file <none>
I20260926 06:53:00.732192 4386 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:00.732236 4386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:00.732383 4386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:00.733345 4386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "ae93d76bb675485384f7a4e86d73055f"
format_stamp: "Formatted at 2026-09-26 06:53:00 on dist-test-slave-ws74"
server_key: "532ace8d025e5f634ad26747b4dcaa80"
server_key_iv: "08bbb2076f4a94c7c89d493567281dfa"
server_key_version: "encryptionkey@0"
I20260926 06:53:00.733769 4386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "ae93d76bb675485384f7a4e86d73055f"
format_stamp: "Formatted at 2026-09-26 06:53:00 on dist-test-slave-ws74"
server_key: "532ace8d025e5f634ad26747b4dcaa80"
server_key_iv: "08bbb2076f4a94c7c89d493567281dfa"
server_key_version: "encryptionkey@0"
I20260926 06:53:00.734997 4386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:00.735639 4401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:00.735798 4386 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:00.735891 4386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "ae93d76bb675485384f7a4e86d73055f"
format_stamp: "Formatted at 2026-09-26 06:53:00 on dist-test-slave-ws74"
server_key: "532ace8d025e5f634ad26747b4dcaa80"
server_key_iv: "08bbb2076f4a94c7c89d493567281dfa"
server_key_version: "encryptionkey@0"
I20260926 06:53:00.735968 4386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:00.749236 4386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:00.750942 4386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:00.751106 4386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:00.751343 4386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:00.751636 4386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:00.751695 4386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:00.751729 4386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:00.751756 4386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:00.757196 4386 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:44227
I20260926 06:53:00.757264 4514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:44227 every 8 connection(s)
I20260926 06:53:00.757586 4386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:53:00.761328 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 4386
I20260926 06:53:00.761454 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:53:00.761653 767 external_mini_cluster.cc:1428] Setting key 7900e4a72874754960f84d6d9ef680aa
Sep 26 06:53:00 dist-test-slave-ws74 krb5kdc[4279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405580, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:00.767294 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:00.758317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:34811 (local address 127.0.191.254:42759)
0926 06:53:00.758401 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:00.758403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:00.758588 (+ 185us) server_negotiation.cc:407] Connection header received
0926 06:53:00.759510 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:00.759511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:00.759523 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:00.759586 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:00.760189 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:00.760685 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:00.761753 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:00.762004 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:00.762069 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:00.764220 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:00.764222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:00.764222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:00.764229 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:00.765772 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:00.766257 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:00.766257 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:00.766258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:00.766308 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:00.766586 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:00.766587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:00.766587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:00.766686 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:00.766734 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:00.767112 (+ 378us) server_negotiation.cc:299] Negotiation successful
0926 06:53:00.767201 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 06:53:00.767516 4517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:00.758296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:42759 (local address 127.0.191.193:34811)
0926 06:53:00.758471 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:00.758504 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:00.759399 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:00.759659 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:00.759663 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:00.759915 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:00.760084 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:00.760088 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:00.760780 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:00.760781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:00.761655 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:00.761657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:00.762083 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:00.762084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:00.762253 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:00.762806 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:00.762822 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:00.764128 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:00.765848 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:00.765855 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:00.765857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:00.766174 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:00.766372 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:00.766373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:00.766373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:00.766517 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:00.766751 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:00.766757 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:00.766925 (+ 168us) client_negotiation.cc:769] Sending connection context
0926 06:53:00.767050 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 06:53:00.767237 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260926 06:53:00.768050 4515 heartbeater.cc:347] Connected to a master server at 127.0.191.254:42759
I20260926 06:53:00.768170 4515 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:00.768330 4515 heartbeater.cc:511] Master 127.0.191.254:42759 requested a full tablet report, sending...
I20260926 06:53:00.768882 4324 ts_manager.cc:194] Registered new tserver with Master: ae93d76bb675485384f7a4e86d73055f (127.0.191.193:44227)
I20260926 06:53:00.769869 4324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:34811
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:53:00.786314 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42759
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:44303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:00.863914 4520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:00.864145 4520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:00.864192 4520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:00.865325 4520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:00.865413 4520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:00.865473 4520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:53:00.866915 4520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:44303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:00.867198 4520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:00.867457 4520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:00.872409 4525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:00.872409 4528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:00.872439 4526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:00.872728 4520 server_base.cc:1034] running on GCE node
I20260926 06:53:00.872969 4520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:00.873302 4520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:00.874485 4520 hybrid_clock.cc:648] HybridClock initialized: now 1790405580874493 us; error 50 us; skew 500 ppm
Sep 26 06:53:00 dist-test-slave-ws74 krb5kdc[4279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405580, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:00.876926 4520 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:53:00.877348 4520 webserver.cc:460] Webserver started at http://127.0.191.194:39113/ using document root <none> and password file <none>
I20260926 06:53:00.877631 4520 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:00.877676 4520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:00.877815 4520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:00.879007 4520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "d3a391becf2445279d5300428f3fbbdb"
format_stamp: "Formatted at 2026-09-26 06:53:00 on dist-test-slave-ws74"
server_key: "557330f3352bfea4284b3916817a03e5"
server_key_iv: "e65ec7577826995be332c17485a4f409"
server_key_version: "encryptionkey@0"
I20260926 06:53:00.879376 4520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "d3a391becf2445279d5300428f3fbbdb"
format_stamp: "Formatted at 2026-09-26 06:53:00 on dist-test-slave-ws74"
server_key: "557330f3352bfea4284b3916817a03e5"
server_key_iv: "e65ec7577826995be332c17485a4f409"
server_key_version: "encryptionkey@0"
I20260926 06:53:00.880743 4520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:53:00.881448 4535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:00.881613 4520 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:00.881697 4520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "d3a391becf2445279d5300428f3fbbdb"
format_stamp: "Formatted at 2026-09-26 06:53:00 on dist-test-slave-ws74"
server_key: "557330f3352bfea4284b3916817a03e5"
server_key_iv: "e65ec7577826995be332c17485a4f409"
server_key_version: "encryptionkey@0"
I20260926 06:53:00.881747 4520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:00.912604 4520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:00.914534 4520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:00.914644 4520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:00.914829 4520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:00.915146 4520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:00.915182 4520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:00.915227 4520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:00.915254 4520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:00.920951 4520 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:40623
I20260926 06:53:00.921028 4648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:40623 every 8 connection(s)
I20260926 06:53:00.921355 4520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:53:00 dist-test-slave-ws74 krb5kdc[4279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405580, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:00.930169 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:00.922107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:56071 (local address 127.0.191.254:42759)
0926 06:53:00.922170 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:00.922172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:00.922397 (+ 225us) server_negotiation.cc:407] Connection header received
0926 06:53:00.923392 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:00.923393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:00.923402 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:00.923457 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:00.924018 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:00.924493 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:00.925612 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:00.925794 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:00.925846 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:00.927881 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:00.927882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:00.927882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:00.927889 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:00.929032 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:00.929413 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:00.929414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:00.929415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:00.929470 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:00.929687 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:00.929688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:00.929688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:00.929782 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:00.929830 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:00.930047 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 06:53:00.930103 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 06:53:00.930330 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:00.922091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:42759 (local address 127.0.191.194:56071)
0926 06:53:00.922261 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:00.922301 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:00.923303 (+ 1002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:00.923510 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:00.923513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:00.923757 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:00.923944 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:00.923948 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:00.924587 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:00.924587 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:00.925545 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:00.925547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:00.925862 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:00.925863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:00.925968 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:00.926477 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:00.926492 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:00.927799 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:00.929099 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:00.929104 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:00.929106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:00.929313 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:00.929527 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:00.929528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:00.929528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:00.929627 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:00.929835 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:00.929839 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:00.929945 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 06:53:00.930023 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 06:53:00.930132 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":103,"threads_started":1}
I20260926 06:53:00.930871 4649 heartbeater.cc:347] Connected to a master server at 127.0.191.254:42759
I20260926 06:53:00.930926 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 4520
I20260926 06:53:00.930972 4649 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:00.931046 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:53:00.931147 4649 heartbeater.cc:511] Master 127.0.191.254:42759 requested a full tablet report, sending...
I20260926 06:53:00.931282 767 external_mini_cluster.cc:1428] Setting key 7f591ad91f01d48e0261133cab5029cf
I20260926 06:53:00.931563 4324 ts_manager.cc:194] Registered new tserver with Master: d3a391becf2445279d5300428f3fbbdb (127.0.191.194:40623)
I20260926 06:53:00.932087 4324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:56071
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:53:00.956513 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42759
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:44303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:01.035701 4654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:01.035908 4654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:01.035933 4654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:01.037107 4654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:01.037158 4654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:01.037186 4654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:53:01.038689 4654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:44303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:01.039003 4654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:01.039240 4654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:01.044281 4662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:01.044281 4659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:01.044338 4660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:01.044436 4654 server_base.cc:1034] running on GCE node
I20260926 06:53:01.044679 4654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:01.044921 4654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:01.046083 4654 hybrid_clock.cc:648] HybridClock initialized: now 1790405581046068 us; error 26 us; skew 500 ppm
Sep 26 06:53:01 dist-test-slave-ws74 krb5kdc[4279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405581, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:01.048512 4654 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:53:01.048890 4654 webserver.cc:460] Webserver started at http://127.0.191.195:36317/ using document root <none> and password file <none>
I20260926 06:53:01.049073 4654 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:01.049113 4654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:01.049201 4654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:01.050264 4654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "c53a40a944cb42ffbb1a74f4d7a001ac"
format_stamp: "Formatted at 2026-09-26 06:53:01 on dist-test-slave-ws74"
server_key: "a7d24e54b11f21f07292b7827ba9cae7"
server_key_iv: "cc1b909f858250ae4ceb969c7d38702e"
server_key_version: "encryptionkey@0"
I20260926 06:53:01.050558 4654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "c53a40a944cb42ffbb1a74f4d7a001ac"
format_stamp: "Formatted at 2026-09-26 06:53:01 on dist-test-slave-ws74"
server_key: "a7d24e54b11f21f07292b7827ba9cae7"
server_key_iv: "cc1b909f858250ae4ceb969c7d38702e"
server_key_version: "encryptionkey@0"
I20260926 06:53:01.051780 4654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:53:01.052424 4669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:01.052573 4654 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:53:01.052659 4654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "c53a40a944cb42ffbb1a74f4d7a001ac"
format_stamp: "Formatted at 2026-09-26 06:53:01 on dist-test-slave-ws74"
server_key: "a7d24e54b11f21f07292b7827ba9cae7"
server_key_iv: "cc1b909f858250ae4ceb969c7d38702e"
server_key_version: "encryptionkey@0"
I20260926 06:53:01.052712 4654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:01.063899 4654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:01.066224 4654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:01.066399 4654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:01.066656 4654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:01.067015 4654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:01.067065 4654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:01.067108 4654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:01.067152 4654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:01.074846 4654 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:39101
I20260926 06:53:01.074909 4782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:39101 every 8 connection(s)
I20260926 06:53:01.075253 4654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:53:01.080812 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 4654
I20260926 06:53:01.080927 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:53:01.081136 767 external_mini_cluster.cc:1428] Setting key 8df8647e9b350bda58b89da85183e0cd
Sep 26 06:53:01 dist-test-slave-ws74 krb5kdc[4279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405581, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:01.085326 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:01.076686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:57941 (local address 127.0.191.254:42759)
0926 06:53:01.076735 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:01.076737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:01.076747 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:53:01.077776 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:01.077777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:01.077808 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:01.077861 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:01.078534 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:01.079037 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:01.080117 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:01.080313 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:01.080400 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:01.082754 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:01.082755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:01.082756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:01.082768 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:01.083917 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:01.084354 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:01.084356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:01.084356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:01.084414 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:01.084678 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:01.084679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:01.084679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:01.084786 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:01.084856 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:01.085134 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 06:53:01.085239 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:53:01.085507 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:01.076281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:42759 (local address 127.0.191.195:57941)
0926 06:53:01.076444 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:01.076485 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:01.077644 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:01.077929 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:01.077933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:01.078186 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:01.078445 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:01.078450 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:01.079158 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:01.079159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:01.080021 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:01.080024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:01.080408 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:01.080409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:01.080537 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:01.081085 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:01.081097 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:01.082648 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:01.084011 (+ 1363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:01.084017 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:01.084020 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:01.084276 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:01.084480 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:01.084481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:01.084481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:01.084603 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:01.084862 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:01.084869 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:01.084997 (+ 128us) client_negotiation.cc:769] Sending connection context
0926 06:53:01.085107 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 06:53:01.085257 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260926 06:53:01.085947 4783 heartbeater.cc:347] Connected to a master server at 127.0.191.254:42759
I20260926 06:53:01.086015 4783 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:01.086196 4783 heartbeater.cc:511] Master 127.0.191.254:42759 requested a full tablet report, sending...
I20260926 06:53:01.086567 4324 ts_manager.cc:194] Registered new tserver with Master: c53a40a944cb42ffbb1a74f4d7a001ac (127.0.191.195:39101)
I20260926 06:53:01.086935 4324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:57941
I20260926 06:53:01.093021 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 06:53:01 dist-test-slave-ws74 krb5kdc[4279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 06:53:01 dist-test-slave-ws74 krb5kdc[4279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.193@KRBTEST.COM
I20260926 06:53:01.115396 4790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:01.107704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39552 (local address 127.0.191.193:44227)
0926 06:53:01.107876 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:01.107882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:01.107893 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:53:01.107926 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:01.107928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:01.107941 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:01.108024 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:01.108586 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:01.109296 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:01.110108 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:01.110438 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:01.110517 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:01.112614 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:01.112621 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:01.112622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:01.112628 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:01.114208 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:01.114584 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:01.114587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:01.114587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:01.114651 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:01.114855 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:01.114856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:01.114856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:01.114985 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:01.115130 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:01.115201 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 06:53:01.115281 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
Sep 26 06:53:01 dist-test-slave-ws74 krb5kdc[4279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:01.123453 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:01.116645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39258 (local address 127.0.191.254:42759)
0926 06:53:01.116708 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:01.116710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:01.116741 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:53:01.116815 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:01.116816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:01.116826 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:01.116881 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:01.117344 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:01.117848 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:01.118532 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:01.118738 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:01.118813 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:01.120860 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:01.120861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:01.120862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:01.120869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:01.122285 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:01.122648 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:01.122649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:01.122650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:01.122702 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:01.122908 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:01.122909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:01.122910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:01.123062 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:01.123170 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:01.123278 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 06:53:01.123352 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260926 06:53:01.123713 4324 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39258
I20260926 06:53:01.124101 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 4386
I20260926 06:53:01.128399 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 4520
I20260926 06:53:01.132489 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 4654
I20260926 06:53:01.136364 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 4292
2026-09-26T06:53:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1836 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:53:01 dist-test-slave-ws74 krb5kdc[4795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:53:01 dist-test-slave-ws74 krb5kdc[4795](info): set up 2 sockets
Sep 26 06:53:01 dist-test-slave-ws74 krb5kdc[4795](info): commencing operation
krb5kdc: starting...
W20260926 06:53:02.161149 767 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:53:02 dist-test-slave-ws74 krb5kdc[4795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:53:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:53:02.275422 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:38707
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:32881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:38707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:02.352744 4808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:02.352934 4808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:02.352955 4808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:02.354154 4808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:02.354195 4808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:02.354209 4808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:02.354221 4808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:02.354233 4808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:02.355654 4808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:32881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:38707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:38707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:02.355839 4808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:02.355996 4808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:02.360560 4814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:02.360565 4813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:02.360580 4808 server_base.cc:1034] running on GCE node
W20260926 06:53:02.360577 4816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:02.361017 4808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:02.361248 4808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:02.362411 4808 hybrid_clock.cc:648] HybridClock initialized: now 1790405582362395 us; error 34 us; skew 500 ppm
Sep 26 06:53:02 dist-test-slave-ws74 krb5kdc[4795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:02.364874 4808 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:53:02.365154 4808 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:02.365195 4808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:02.365280 4808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:02.366278 4808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "41921f48f66b4698bbf875d0f9546eb7"
format_stamp: "Formatted at 2026-09-26 06:53:02 on dist-test-slave-ws74"
server_key: "07778a229a84a89cd59650302aeb9a75"
server_key_iv: "b146e7480c978dc3b3e85a87f9b9bd56"
server_key_version: "encryptionkey@0"
I20260926 06:53:02.366556 4808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "41921f48f66b4698bbf875d0f9546eb7"
format_stamp: "Formatted at 2026-09-26 06:53:02 on dist-test-slave-ws74"
server_key: "07778a229a84a89cd59650302aeb9a75"
server_key_iv: "b146e7480c978dc3b3e85a87f9b9bd56"
server_key_version: "encryptionkey@0"
I20260926 06:53:02.367695 4808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:53:02.368377 4822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:02.368522 4808 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:02.368602 4808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "41921f48f66b4698bbf875d0f9546eb7"
format_stamp: "Formatted at 2026-09-26 06:53:02 on dist-test-slave-ws74"
server_key: "07778a229a84a89cd59650302aeb9a75"
server_key_iv: "b146e7480c978dc3b3e85a87f9b9bd56"
server_key_version: "encryptionkey@0"
I20260926 06:53:02.368654 4808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:02.380491 4808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:02.382432 4808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:02.382532 4808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:02.386214 4808 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:38707
I20260926 06:53:02.386240 4874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:38707 every 8 connection(s)
I20260926 06:53:02.386502 4808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:02.387411 4875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:02.389562 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 4808
I20260926 06:53:02.389643 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:02.389659 4875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7: Bootstrap starting.
I20260926 06:53:02.389834 767 external_mini_cluster.cc:1428] Setting key 2d5da008b0ae82b6ffbc7a1a00c1b05f
I20260926 06:53:02.390359 4875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:02.390669 4875 log.cc:824] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:02.391332 4875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7: No bootstrap required, opened a new log
I20260926 06:53:02.393191 4875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41921f48f66b4698bbf875d0f9546eb7" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38707 } }
I20260926 06:53:02.393353 4875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:02.393414 4875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41921f48f66b4698bbf875d0f9546eb7, State: Initialized, Role: FOLLOWER
I20260926 06:53:02.393538 4875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41921f48f66b4698bbf875d0f9546eb7" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38707 } }
I20260926 06:53:02.393622 4875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:02.393651 4875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:02.393687 4875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:02.394415 4875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41921f48f66b4698bbf875d0f9546eb7" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38707 } }
I20260926 06:53:02.394526 4875 leader_election.cc:302] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41921f48f66b4698bbf875d0f9546eb7; no voters:
I20260926 06:53:02.394693 4875 leader_election.cc:288] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:02.394881 4881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:02.394984 4875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:02.395016 4881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [term 1 LEADER]: Becoming Leader. State: Replica: 41921f48f66b4698bbf875d0f9546eb7, State: Running, Role: LEADER
I20260926 06:53:02.395097 4881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41921f48f66b4698bbf875d0f9546eb7" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38707 } }
I20260926 06:53:02.395406 4881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41921f48f66b4698bbf875d0f9546eb7. Latest consensus state: current_term: 1 leader_uuid: "41921f48f66b4698bbf875d0f9546eb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41921f48f66b4698bbf875d0f9546eb7" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38707 } } }
I20260926 06:53:02.395498 4881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:02.395417 4882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41921f48f66b4698bbf875d0f9546eb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41921f48f66b4698bbf875d0f9546eb7" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38707 } } }
I20260926 06:53:02.395768 4882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7 [sys.catalog]: This master's current role is: LEADER
Sep 26 06:53:02 dist-test-slave-ws74 krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
W20260926 06:53:02.397096 4895 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:53:02.397148 4895 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:53:02.397222 4888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:02.397396 4888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:02.399641 4888 catalog_manager.cc:1347] Generated new cluster ID: 57279496c10240f8af107520c2c14937
I20260926 06:53:02.399693 4888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:53:02.400349 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.390316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55132 (local address 127.0.191.254:38707)
0926 06:53:02.390523 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:02.390531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:02.390564 (+ 33us) server_negotiation.cc:407] Connection header received
0926 06:53:02.391002 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:02.391008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:02.391052 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:02.391201 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:02.391856 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.392706 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.393507 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.394116 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.394242 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.397024 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:02.397028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:02.397028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:02.397047 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:02.398736 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:02.399128 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:02.399131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:02.399131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:02.399243 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:02.399452 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:02.399453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:02.399453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:02.399766 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:02.399911 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:02.400025 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 06:53:02.400152 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":99,"threads_started":1}
I20260926 06:53:02.412098 4888 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:02.412662 4888 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:02.420087 4888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7: Generated new TSK 0
I20260926 06:53:02.420297 4888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:53:02.455740 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38707
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:32881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:02.555430 4901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:02.555621 4901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:02.555645 4901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:02.556752 4901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:02.556785 4901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:02.556813 4901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:53:02.558363 4901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:32881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:02.558545 4901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:02.558713 4901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:02.563557 4907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:02.563649 4909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:02.563664 4906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:02.563740 4901 server_base.cc:1034] running on GCE node
I20260926 06:53:02.564052 4901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:02.564296 4901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:02.565440 4901 hybrid_clock.cc:648] HybridClock initialized: now 1790405582565420 us; error 35 us; skew 500 ppm
Sep 26 06:53:02 dist-test-slave-ws74 krb5kdc[4795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:02.567966 4901 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:53:02.568325 4901 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:02.568377 4901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:02.568463 4901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:02.569487 4901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "0bc51e45168e47ddbbff7aa798a4d94e"
format_stamp: "Formatted at 2026-09-26 06:53:02 on dist-test-slave-ws74"
server_key: "23960004193e3d1c66e038fa63375c09"
server_key_iv: "9179680fe830aa34851157686307fe0b"
server_key_version: "encryptionkey@0"
I20260926 06:53:02.569784 4901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "0bc51e45168e47ddbbff7aa798a4d94e"
format_stamp: "Formatted at 2026-09-26 06:53:02 on dist-test-slave-ws74"
server_key: "23960004193e3d1c66e038fa63375c09"
server_key_iv: "9179680fe830aa34851157686307fe0b"
server_key_version: "encryptionkey@0"
I20260926 06:53:02.570986 4901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:02.571751 4915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:02.571898 4901 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:02.571980 4901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "0bc51e45168e47ddbbff7aa798a4d94e"
format_stamp: "Formatted at 2026-09-26 06:53:02 on dist-test-slave-ws74"
server_key: "23960004193e3d1c66e038fa63375c09"
server_key_iv: "9179680fe830aa34851157686307fe0b"
server_key_version: "encryptionkey@0"
I20260926 06:53:02.572062 4901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:02.603462 4901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:02.605597 4901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:02.605721 4901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:02.605903 4901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:02.606225 4901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:02.606261 4901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:02.606283 4901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:02.606295 4901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:02.611658 4901 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:39787
I20260926 06:53:02.611765 5028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:39787 every 8 connection(s)
I20260926 06:53:02.611994 4901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:53:02 dist-test-slave-ws74 krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:02.620330 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.612668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:52793 (local address 127.0.191.254:38707)
0926 06:53:02.612728 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:02.612730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:02.612968 (+ 238us) server_negotiation.cc:407] Connection header received
0926 06:53:02.613452 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:02.613453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:02.613464 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:02.613518 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:02.614366 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.614873 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.615783 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.615979 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.616027 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.618116 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:02.618117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:02.618117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:02.618122 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:02.619279 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:02.619606 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:02.619607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:02.619607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:02.619652 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:02.619858 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:02.619859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:02.619859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:02.619948 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:02.619995 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:02.620208 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 06:53:02.620269 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 06:53:02.620503 5031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.612657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:38707 (local address 127.0.191.193:52793)
0926 06:53:02.612852 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:02.612886 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:02.613332 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:02.613589 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:02.613592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:02.613838 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:02.614292 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:02.614297 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.614948 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.614948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:02.615708 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:02.615709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.616046 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.616047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:02.616139 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.616643 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:02.616651 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:02.618041 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:02.619346 (+ 1305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:02.619352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:02.619354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:02.619547 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:02.619706 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:02.619706 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:02.619707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:02.619807 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:02.620000 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:02.620003 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:02.620104 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 06:53:02.620179 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:53:02.620286 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
I20260926 06:53:02.621070 5029 heartbeater.cc:347] Connected to a master server at 127.0.191.254:38707
I20260926 06:53:02.621174 5029 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:02.621311 5029 heartbeater.cc:511] Master 127.0.191.254:38707 requested a full tablet report, sending...
I20260926 06:53:02.621522 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 4901
I20260926 06:53:02.621639 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:53:02.621829 767 external_mini_cluster.cc:1428] Setting key 09bc2a2e331417364cca12d0491d7623
I20260926 06:53:02.621920 4839 ts_manager.cc:194] Registered new tserver with Master: 0bc51e45168e47ddbbff7aa798a4d94e (127.0.191.193:39787)
I20260926 06:53:02.622872 4839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:52793
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:53:02.645989 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38707
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:32881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:02.724706 5034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:02.724897 5034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:02.724920 5034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:02.726186 5034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:02.726230 5034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:02.726256 5034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:53:02.727633 5034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:32881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:02.727795 5034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:02.727952 5034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:02.732775 5040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:02.732782 5042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:02.732810 5034 server_base.cc:1034] running on GCE node
W20260926 06:53:02.732782 5039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:02.733177 5034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:02.733358 5034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:02.734586 5034 hybrid_clock.cc:648] HybridClock initialized: now 1790405582734571 us; error 30 us; skew 500 ppm
Sep 26 06:53:02 dist-test-slave-ws74 krb5kdc[4795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:02.737044 5034 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:53:02.737466 5034 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:02.737516 5034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:02.737671 5034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:02.738731 5034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "ebd42dd29def4b20a6b84b2dfccf3d34"
format_stamp: "Formatted at 2026-09-26 06:53:02 on dist-test-slave-ws74"
server_key: "58ad7d53ce5eed2747e15c70072531b8"
server_key_iv: "e5f4d41e078d2b6a41572f8de3b7d230"
server_key_version: "encryptionkey@0"
I20260926 06:53:02.739095 5034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "ebd42dd29def4b20a6b84b2dfccf3d34"
format_stamp: "Formatted at 2026-09-26 06:53:02 on dist-test-slave-ws74"
server_key: "58ad7d53ce5eed2747e15c70072531b8"
server_key_iv: "e5f4d41e078d2b6a41572f8de3b7d230"
server_key_version: "encryptionkey@0"
I20260926 06:53:02.740386 5034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:53:02.741113 5048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:02.741266 5034 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:02.741359 5034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "ebd42dd29def4b20a6b84b2dfccf3d34"
format_stamp: "Formatted at 2026-09-26 06:53:02 on dist-test-slave-ws74"
server_key: "58ad7d53ce5eed2747e15c70072531b8"
server_key_iv: "e5f4d41e078d2b6a41572f8de3b7d230"
server_key_version: "encryptionkey@0"
I20260926 06:53:02.741461 5034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:02.767320 5034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:02.769125 5034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:02.769243 5034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:02.769495 5034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:02.769759 5034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:02.769790 5034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:02.769812 5034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:02.769824 5034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:02.775847 5034 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:46873
I20260926 06:53:02.775936 5161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:46873 every 8 connection(s)
I20260926 06:53:02.776185 5034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:53:02.780603 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 5034
I20260926 06:53:02.780731 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:53:02.780929 767 external_mini_cluster.cc:1428] Setting key 72875779e474c70d6dcb765a2d0f1b92
Sep 26 06:53:02 dist-test-slave-ws74 krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:02.786098 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.776851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:38707 (local address 127.0.191.194:48989)
0926 06:53:02.776981 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:02.777025 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:02.777607 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:02.777860 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:02.777864 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:02.778115 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:02.778572 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:02.778577 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.779264 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.779266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:02.780046 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:02.780048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.780490 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.780492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:02.780636 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.781139 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:02.781148 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:02.782981 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:02.784760 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:02.784766 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:02.784768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:02.784971 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:02.785183 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:02.785183 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:02.785183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:02.785317 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:02.785641 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:02.785645 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:02.785760 (+ 115us) client_negotiation.cc:769] Sending connection context
0926 06:53:02.785841 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 06:53:02.785949 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260926 06:53:02.786111 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.776888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:48989 (local address 127.0.191.254:38707)
0926 06:53:02.776932 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:02.776935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:02.777116 (+ 181us) server_negotiation.cc:407] Connection header received
0926 06:53:02.777721 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:02.777722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:02.777736 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:02.777782 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:02.778668 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.779171 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.780169 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.780373 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.780484 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.783115 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:02.783117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:02.783117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:02.783126 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:02.784682 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:02.785049 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:02.785050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:02.785050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:02.785116 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:02.785411 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:02.785413 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:02.785413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:02.785572 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:02.785648 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:02.785889 (+ 241us) server_negotiation.cc:299] Negotiation successful
0926 06:53:02.785976 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:53:02.786566 5162 heartbeater.cc:347] Connected to a master server at 127.0.191.254:38707
I20260926 06:53:02.786667 5162 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:02.786829 5162 heartbeater.cc:511] Master 127.0.191.254:38707 requested a full tablet report, sending...
I20260926 06:53:02.787184 4839 ts_manager.cc:194] Registered new tserver with Master: ebd42dd29def4b20a6b84b2dfccf3d34 (127.0.191.194:46873)
I20260926 06:53:02.787577 4839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:48989
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:53:02.806607 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38707
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:32881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:02.884927 5167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:02.885154 5167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:02.885237 5167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:02.886374 5167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:02.886438 5167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:02.886483 5167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:53:02.887941 5167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:32881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:02.888206 5167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:02.888430 5167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:02.893302 5173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:02.893313 5172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:02.893313 5175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:02.893330 5167 server_base.cc:1034] running on GCE node
I20260926 06:53:02.893738 5167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:02.893965 5167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:02.895104 5167 hybrid_clock.cc:648] HybridClock initialized: now 1790405582895106 us; error 47 us; skew 500 ppm
Sep 26 06:53:02 dist-test-slave-ws74 krb5kdc[4795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:02.897897 5167 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:53:02.898250 5167 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:02.898324 5167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:02.898471 5167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:02.899492 5167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "75552adb4d054382b0fcd1de43f64e55"
format_stamp: "Formatted at 2026-09-26 06:53:02 on dist-test-slave-ws74"
server_key: "0278b8a957d8ed8354ccc85c4915c271"
server_key_iv: "2b01610ee6d0a34e8f08a56b61c41018"
server_key_version: "encryptionkey@0"
I20260926 06:53:02.899840 5167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "75552adb4d054382b0fcd1de43f64e55"
format_stamp: "Formatted at 2026-09-26 06:53:02 on dist-test-slave-ws74"
server_key: "0278b8a957d8ed8354ccc85c4915c271"
server_key_iv: "2b01610ee6d0a34e8f08a56b61c41018"
server_key_version: "encryptionkey@0"
I20260926 06:53:02.901170 5167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:53:02.901903 5181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:02.902076 5167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:02.902182 5167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "75552adb4d054382b0fcd1de43f64e55"
format_stamp: "Formatted at 2026-09-26 06:53:02 on dist-test-slave-ws74"
server_key: "0278b8a957d8ed8354ccc85c4915c271"
server_key_iv: "2b01610ee6d0a34e8f08a56b61c41018"
server_key_version: "encryptionkey@0"
I20260926 06:53:02.902254 5167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:02.915012 5167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:02.917281 5167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:02.917477 5167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:02.917709 5167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:02.918007 5167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:02.918061 5167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:02.918099 5167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:02.918133 5167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:02.924062 5167 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:44277
I20260926 06:53:02.924131 5294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:44277 every 8 connection(s)
I20260926 06:53:02.924386 5167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:53:02 dist-test-slave-ws74 krb5kdc[4795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:02.930907 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 5167
I20260926 06:53:02.931000 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:53:02.931174 767 external_mini_cluster.cc:1428] Setting key 285292837df2c7a97ee6e276633fe85b
I20260926 06:53:02.933301 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.925064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:52651 (local address 127.0.191.254:38707)
0926 06:53:02.925138 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:02.925140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:02.925275 (+ 135us) server_negotiation.cc:407] Connection header received
0926 06:53:02.925785 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:02.925786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:02.925797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:02.925848 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:02.926685 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.927164 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.928159 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.928334 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.928382 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.930599 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:02.930601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:02.930601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:02.930606 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:02.932209 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:02.932553 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:02.932554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:02.932554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:02.932600 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:02.932801 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:02.932802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:02.932802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:02.932905 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:02.932951 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:02.933191 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 06:53:02.933242 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 06:53:02.933485 5297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.925019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:38707 (local address 127.0.191.195:52651)
0926 06:53:02.925166 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:02.925199 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:02.925702 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:02.925901 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:02.925904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:02.926140 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:02.926611 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:02.926617 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.927259 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.927261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:02.928054 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:02.928056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.928402 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.928403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:02.928508 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.929027 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:02.929035 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:02.930516 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:02.932275 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:02.932279 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:02.932281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:02.932486 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:02.932655 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:02.932656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:02.932656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:02.932748 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:02.932954 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:02.932957 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:02.933056 (+ 99us) client_negotiation.cc:769] Sending connection context
0926 06:53:02.933166 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 06:53:02.933277 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":91,"threads_started":1}
I20260926 06:53:02.933907 5295 heartbeater.cc:347] Connected to a master server at 127.0.191.254:38707
I20260926 06:53:02.933993 5295 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:02.934094 5295 heartbeater.cc:511] Master 127.0.191.254:38707 requested a full tablet report, sending...
I20260926 06:53:02.934373 4839 ts_manager.cc:194] Registered new tserver with Master: 75552adb4d054382b0fcd1de43f64e55 (127.0.191.195:44277)
I20260926 06:53:02.934684 4839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:52651
I20260926 06:53:02.942674 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:53:02.950007 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.943900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55136 (local address 127.0.191.254:38707)
0926 06:53:02.943956 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:02.943958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:02.944065 (+ 107us) server_negotiation.cc:407] Connection header received
0926 06:53:02.944168 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:02.944169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:02.944179 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:02.944247 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:02.944798 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.945310 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.946035 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.946224 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.946298 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.947454 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:02.947455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:02.947456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:02.947462 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:02.948817 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:02.949171 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:02.949172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:02.949173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:02.949232 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:02.949471 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:02.949472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:02.949473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:02.949615 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:02.949726 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:02.949834 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 06:53:02.949908 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:53:02.951638 4839 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:53:02.953137 4839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:53:02.959654 5310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.955399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39787 (local address 127.0.0.1:45760)
0926 06:53:02.955760 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:02.955770 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:02.955817 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:02.955992 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:02.955995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:02.955997 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:02.956062 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:02.956064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.957483 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.957484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:02.958458 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:02.958460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.959336 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.959337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:02.959424 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.959437 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 06:53:02.959508 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 06:53:02.959569 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":88,"threads_started":1}
I20260926 06:53:02.959671 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.955500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45760 (local address 127.0.191.193:39787)
0926 06:53:02.955792 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:02.955798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:02.955808 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:53:02.955884 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:02.955886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:02.955901 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:02.955978 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:02.956155 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.957343 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.958535 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.959245 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.959324 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.959386 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:02.959541 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 06:53:02.959579 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":79,"threads_started":1}
I20260926 06:53:02.959757 5308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.955490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49844 (local address 127.0.191.194:46873)
0926 06:53:02.955934 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:02.955937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:02.955946 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:02.955984 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:02.955986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:02.956001 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:02.956094 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:02.956246 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.957139 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.958644 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.959296 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.959560 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.959612 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:02.959634 (+ 22us) server_negotiation.cc:299] Negotiation successful
0926 06:53:02.959670 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":75,"threads_started":1}
I20260926 06:53:02.959928 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.955396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:46873 (local address 127.0.0.1:49844)
0926 06:53:02.955550 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:02.955602 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:02.955662 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:02.956105 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:02.956106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:02.956108 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:02.956164 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:02.956166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.957238 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.957239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:02.958555 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:02.958557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.959386 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.959387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:02.959449 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.959452 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:02.959511 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 06:53:02.959858 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":293,"thread_start_us":44,"threads_started":1}
I20260926 06:53:02.960000 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.955730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39158 (local address 127.0.191.195:44277)
0926 06:53:02.955888 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:02.955893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:02.956327 (+ 434us) server_negotiation.cc:407] Connection header received
0926 06:53:02.956363 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:02.956365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:02.956379 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:02.956451 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:02.956578 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.957632 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.958840 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.959348 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.959417 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.959471 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:02.959691 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 06:53:02.959915 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260926 06:53:02.960059 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:02.955574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:44277 (local address 127.0.0.1:39158)
0926 06:53:02.956215 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:02.956226 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:02.956274 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:02.956465 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:02.956466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:02.956468 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:02.956503 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:02.956504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.957725 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.957726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:02.958763 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:02.958764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:02.959505 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:02.959506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:02.959559 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:02.959562 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:02.959608 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 06:53:02.959640 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":64,"threads_started":1}
I20260926 06:53:02.960563 4963 tablet_service.cc:1467] Processing CreateTablet for tablet a6fed4b381cc45b5903c4f1a19d7a106 (DEFAULT_TABLE table=test-table [id=168f3fc9e093475fbe1eb34dbaa462e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:02.960716 5096 tablet_service.cc:1467] Processing CreateTablet for tablet a6fed4b381cc45b5903c4f1a19d7a106 (DEFAULT_TABLE table=test-table [id=168f3fc9e093475fbe1eb34dbaa462e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:02.960741 5229 tablet_service.cc:1467] Processing CreateTablet for tablet a6fed4b381cc45b5903c4f1a19d7a106 (DEFAULT_TABLE table=test-table [id=168f3fc9e093475fbe1eb34dbaa462e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:02.960915 4963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6fed4b381cc45b5903c4f1a19d7a106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:02.960979 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6fed4b381cc45b5903c4f1a19d7a106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:02.961091 5229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6fed4b381cc45b5903c4f1a19d7a106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:02.963511 5312 tablet_bootstrap.cc:492] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55: Bootstrap starting.
I20260926 06:53:02.964212 5312 tablet_bootstrap.cc:654] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:02.964385 5314 tablet_bootstrap.cc:492] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34: Bootstrap starting.
I20260926 06:53:02.964857 5312 log.cc:824] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:02.965026 5314 tablet_bootstrap.cc:654] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:02.965471 5313 tablet_bootstrap.cc:492] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e: Bootstrap starting.
I20260926 06:53:02.965608 5314 log.cc:824] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:02.965696 5312 tablet_bootstrap.cc:492] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55: No bootstrap required, opened a new log
I20260926 06:53:02.965758 5312 ts_tablet_manager.cc:1397] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:53:02.966045 5313 tablet_bootstrap.cc:654] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:02.966560 5313 log.cc:824] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:02.967247 5313 tablet_bootstrap.cc:492] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e: No bootstrap required, opened a new log
I20260926 06:53:02.967326 5313 ts_tablet_manager.cc:1397] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 06:53:02.967358 5312 raft_consensus.cc:348] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd42dd29def4b20a6b84b2dfccf3d34" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46873 } } peers { permanent_uuid: "0bc51e45168e47ddbbff7aa798a4d94e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39787 } } peers { permanent_uuid: "75552adb4d054382b0fcd1de43f64e55" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44277 } }
I20260926 06:53:02.967507 5312 raft_consensus.cc:374] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:02.967568 5312 raft_consensus.cc:729] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75552adb4d054382b0fcd1de43f64e55, State: Initialized, Role: FOLLOWER
I20260926 06:53:02.967715 5312 consensus_queue.cc:260] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd42dd29def4b20a6b84b2dfccf3d34" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46873 } } peers { permanent_uuid: "0bc51e45168e47ddbbff7aa798a4d94e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39787 } } peers { permanent_uuid: "75552adb4d054382b0fcd1de43f64e55" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44277 } }
I20260926 06:53:02.968012 5295 heartbeater.cc:503] Master 127.0.191.254:38707 was elected leader, sending a full tablet report...
I20260926 06:53:02.968022 5312 ts_tablet_manager.cc:1428] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:53:02.968523 5314 tablet_bootstrap.cc:492] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34: No bootstrap required, opened a new log
I20260926 06:53:02.968592 5314 ts_tablet_manager.cc:1397] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 06:53:02.969450 5313 raft_consensus.cc:348] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd42dd29def4b20a6b84b2dfccf3d34" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46873 } } peers { permanent_uuid: "0bc51e45168e47ddbbff7aa798a4d94e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39787 } } peers { permanent_uuid: "75552adb4d054382b0fcd1de43f64e55" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44277 } }
I20260926 06:53:02.969606 5313 raft_consensus.cc:374] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:02.969643 5313 raft_consensus.cc:729] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc51e45168e47ddbbff7aa798a4d94e, State: Initialized, Role: FOLLOWER
I20260926 06:53:02.969746 5313 consensus_queue.cc:260] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd42dd29def4b20a6b84b2dfccf3d34" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46873 } } peers { permanent_uuid: "0bc51e45168e47ddbbff7aa798a4d94e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39787 } } peers { permanent_uuid: "75552adb4d054382b0fcd1de43f64e55" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44277 } }
I20260926 06:53:02.970013 5313 ts_tablet_manager.cc:1428] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:53:02.970023 5029 heartbeater.cc:503] Master 127.0.191.254:38707 was elected leader, sending a full tablet report...
I20260926 06:53:02.970659 5314 raft_consensus.cc:348] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd42dd29def4b20a6b84b2dfccf3d34" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46873 } } peers { permanent_uuid: "0bc51e45168e47ddbbff7aa798a4d94e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39787 } } peers { permanent_uuid: "75552adb4d054382b0fcd1de43f64e55" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44277 } }
I20260926 06:53:02.970846 5314 raft_consensus.cc:374] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:02.970893 5314 raft_consensus.cc:729] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebd42dd29def4b20a6b84b2dfccf3d34, State: Initialized, Role: FOLLOWER
I20260926 06:53:02.971012 5314 consensus_queue.cc:260] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd42dd29def4b20a6b84b2dfccf3d34" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46873 } } peers { permanent_uuid: "0bc51e45168e47ddbbff7aa798a4d94e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39787 } } peers { permanent_uuid: "75552adb4d054382b0fcd1de43f64e55" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44277 } }
I20260926 06:53:02.971273 5314 ts_tablet_manager.cc:1428] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:53:02.971302 5162 heartbeater.cc:503] Master 127.0.191.254:38707 was elected leader, sending a full tablet report...
W20260926 06:53:03.026724 5163 tablet.cc:2369] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:53:03.103235 5318 raft_consensus.cc:482] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:53:03.103380 5318 raft_consensus.cc:504] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd42dd29def4b20a6b84b2dfccf3d34" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46873 } } peers { permanent_uuid: "0bc51e45168e47ddbbff7aa798a4d94e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39787 } } peers { permanent_uuid: "75552adb4d054382b0fcd1de43f64e55" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44277 } }
I20260926 06:53:03.103770 5318 leader_election.cc:288] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebd42dd29def4b20a6b84b2dfccf3d34 (127.0.191.194:46873), 0bc51e45168e47ddbbff7aa798a4d94e (127.0.191.193:39787)
I20260926 06:53:03.106813 5308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.104038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:46115 (local address 127.0.191.194:46873)
0926 06:53:03.104085 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.104087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.104098 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:53:03.104161 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.104162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.104172 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.104240 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:03.104436 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.105007 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.106040 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.106454 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.106508 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:03.106515 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.106703 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.106744 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:53:03.106814 5297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.103957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:46873 (local address 127.0.191.195:46115)
0926 06:53:03.104038 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:03.104051 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:03.104098 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:03.104266 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:03.104267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:03.104270 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:03.104347 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:03.104349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.105099 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.105100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:03.105979 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:03.105981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.106525 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.106526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:03.106613 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:03.106616 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:03.106667 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 06:53:03.106712 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":20}
I20260926 06:53:03.106900 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.104064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:52241 (local address 127.0.191.193:39787)
0926 06:53:03.104116 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.104118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.104271 (+ 153us) server_negotiation.cc:407] Connection header received
0926 06:53:03.104333 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.104334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.104343 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.104403 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:03.104557 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.104990 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.106156 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.106522 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.106592 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:03.106600 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.106791 (+ 191us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.106830 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 06:53:03.106956 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.103961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39787 (local address 127.0.191.195:52241)
0926 06:53:03.104196 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:03.104207 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:03.104274 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:03.104419 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:03.104420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:03.104422 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:03.104492 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:03.104493 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.105076 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.105077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:03.106072 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:03.106074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.106606 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.106607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:03.106710 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:03.106714 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:53:03.106768 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 06:53:03.106804 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":135,"thread_start_us":108,"threads_started":1}
I20260926 06:53:03.107182 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6fed4b381cc45b5903c4f1a19d7a106" candidate_uuid: "75552adb4d054382b0fcd1de43f64e55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebd42dd29def4b20a6b84b2dfccf3d34" is_pre_election: true
I20260926 06:53:03.107213 4983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6fed4b381cc45b5903c4f1a19d7a106" candidate_uuid: "75552adb4d054382b0fcd1de43f64e55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc51e45168e47ddbbff7aa798a4d94e" is_pre_election: true
I20260926 06:53:03.107334 4983 raft_consensus.cc:2454] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75552adb4d054382b0fcd1de43f64e55 in term 0.
I20260926 06:53:03.107334 5116 raft_consensus.cc:2454] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75552adb4d054382b0fcd1de43f64e55 in term 0.
I20260926 06:53:03.107568 5184 leader_election.cc:302] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc51e45168e47ddbbff7aa798a4d94e, 75552adb4d054382b0fcd1de43f64e55; no voters:
I20260926 06:53:03.107703 5318 raft_consensus.cc:2781] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:53:03.107785 5318 raft_consensus.cc:482] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:53:03.107820 5318 raft_consensus.cc:3037] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:03.108497 5318 raft_consensus.cc:504] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd42dd29def4b20a6b84b2dfccf3d34" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46873 } } peers { permanent_uuid: "0bc51e45168e47ddbbff7aa798a4d94e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39787 } } peers { permanent_uuid: "75552adb4d054382b0fcd1de43f64e55" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44277 } }
I20260926 06:53:03.108637 5318 leader_election.cc:288] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [CANDIDATE]: Term 1 election: Requested vote from peers ebd42dd29def4b20a6b84b2dfccf3d34 (127.0.191.194:46873), 0bc51e45168e47ddbbff7aa798a4d94e (127.0.191.193:39787)
I20260926 06:53:03.108795 4983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6fed4b381cc45b5903c4f1a19d7a106" candidate_uuid: "75552adb4d054382b0fcd1de43f64e55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc51e45168e47ddbbff7aa798a4d94e"
I20260926 06:53:03.108876 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6fed4b381cc45b5903c4f1a19d7a106" candidate_uuid: "75552adb4d054382b0fcd1de43f64e55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebd42dd29def4b20a6b84b2dfccf3d34"
I20260926 06:53:03.108887 4983 raft_consensus.cc:3037] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:03.108945 5116 raft_consensus.cc:3037] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:03.109826 5116 raft_consensus.cc:2454] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75552adb4d054382b0fcd1de43f64e55 in term 1.
I20260926 06:53:03.109826 4983 raft_consensus.cc:2454] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75552adb4d054382b0fcd1de43f64e55 in term 1.
I20260926 06:53:03.110008 5184 leader_election.cc:302] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc51e45168e47ddbbff7aa798a4d94e, 75552adb4d054382b0fcd1de43f64e55; no voters:
I20260926 06:53:03.110112 5318 raft_consensus.cc:2781] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:03.110275 5318 raft_consensus.cc:686] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [term 1 LEADER]: Becoming Leader. State: Replica: 75552adb4d054382b0fcd1de43f64e55, State: Running, Role: LEADER
I20260926 06:53:03.110378 5318 consensus_queue.cc:237] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd42dd29def4b20a6b84b2dfccf3d34" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46873 } } peers { permanent_uuid: "0bc51e45168e47ddbbff7aa798a4d94e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39787 } } peers { permanent_uuid: "75552adb4d054382b0fcd1de43f64e55" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44277 } }
I20260926 06:53:03.111168 4839 catalog_manager.cc:5495] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75552adb4d054382b0fcd1de43f64e55 (127.0.191.195). New cstate: current_term: 1 leader_uuid: "75552adb4d054382b0fcd1de43f64e55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd42dd29def4b20a6b84b2dfccf3d34" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bc51e45168e47ddbbff7aa798a4d94e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75552adb4d054382b0fcd1de43f64e55" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44277 } health_report { overall_health: HEALTHY } } }
W20260926 06:53:03.112614 5030 tablet.cc:2369] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:53:03.139547 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.137560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39168 (local address 127.0.191.195:44277)
0926 06:53:03.137618 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.137620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.137636 (+ 16us) server_negotiation.cc:407] Connection header received
0926 06:53:03.137667 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.137667 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.137683 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.137748 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:53:03.137826 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.138267 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.138948 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.139115 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.139176 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:03.139285 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:53:03.139374 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:53:03.139431 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.139465 (+ 34us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.139480 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:53:03.141264 5116 raft_consensus.cc:1261] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34 [term 1 FOLLOWER]: Refusing update from remote peer 75552adb4d054382b0fcd1de43f64e55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:03.141273 4983 raft_consensus.cc:1261] T a6fed4b381cc45b5903c4f1a19d7a106 P 0bc51e45168e47ddbbff7aa798a4d94e [term 1 FOLLOWER]: Refusing update from remote peer 75552adb4d054382b0fcd1de43f64e55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:03.141640 5322 consensus_queue.cc:1035] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bc51e45168e47ddbbff7aa798a4d94e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:03.141789 5318 consensus_queue.cc:1035] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebd42dd29def4b20a6b84b2dfccf3d34" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:03.144064 5327 mvcc.cc:205] Tried to move back new op lower bound from 7333501268544057344 to 7333501268420849664. Current Snapshot: MvccSnapshot[applied={T|T < 7333501268544057344}]
I20260926 06:53:03.144546 5326 mvcc.cc:205] Tried to move back new op lower bound from 7333501268544057344 to 7333501268420849664. Current Snapshot: MvccSnapshot[applied={T|T < 7333501268544057344}]
I20260926 06:53:03.146251 4838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:03.146339 4838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:03.147037 4838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41921f48f66b4698bbf875d0f9546eb7: Sending DeleteTablet for 3 replicas of tablet a6fed4b381cc45b5903c4f1a19d7a106
I20260926 06:53:03.147336 4963 tablet_service.cc:1514] Processing DeleteTablet for tablet a6fed4b381cc45b5903c4f1a19d7a106 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:03 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:45760
I20260926 06:53:03.147342 5096 tablet_service.cc:1514] Processing DeleteTablet for tablet a6fed4b381cc45b5903c4f1a19d7a106 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:03 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:49844
I20260926 06:53:03.147353 5229 tablet_service.cc:1514] Processing DeleteTablet for tablet a6fed4b381cc45b5903c4f1a19d7a106 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:03 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:39158
I20260926 06:53:03.147535 5335 tablet_replica.cc:330] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34: stopping tablet replica
I20260926 06:53:03.147619 5335 raft_consensus.cc:2229] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:03.147709 5335 raft_consensus.cc:2258] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:03.147894 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 4901
I20260926 06:53:03.148160 5335 ts_tablet_manager.cc:1905] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:03.150075 5335 ts_tablet_manager.cc:1918] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:03.150138 5335 log.cc:1196] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/a6fed4b381cc45b5903c4f1a19d7a106
I20260926 06:53:03.150413 5335 ts_tablet_manager.cc:1939] T a6fed4b381cc45b5903c4f1a19d7a106 P ebd42dd29def4b20a6b84b2dfccf3d34: Deleting consensus metadata
I20260926 06:53:03.150856 4826 catalog_manager.cc:4841] TS ebd42dd29def4b20a6b84b2dfccf3d34 (127.0.191.194:46873): tablet a6fed4b381cc45b5903c4f1a19d7a106 (table test-table [id=168f3fc9e093475fbe1eb34dbaa462e2]) successfully deleted
I20260926 06:53:03.153533 5337 tablet_replica.cc:330] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55: stopping tablet replica
I20260926 06:53:03.153627 5337 raft_consensus.cc:2229] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:53:03.153715 5337 raft_consensus.cc:2258] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:03.154120 5337 ts_tablet_manager.cc:1905] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 06:53:03.154595 4825 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:39787 (error 108)
I20260926 06:53:03.154850 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 5034
I20260926 06:53:03.155772 5337 ts_tablet_manager.cc:1918] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:03.155841 5337 log.cc:1196] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/a6fed4b381cc45b5903c4f1a19d7a106
I20260926 06:53:03.155843 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.155066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39787 (local address 127.0.0.1:45766)
0926 06:53:03.155758 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:03.155792 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:39787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":675}
I20260926 06:53:03.156092 5337 ts_tablet_manager.cc:1939] T a6fed4b381cc45b5903c4f1a19d7a106 P 75552adb4d054382b0fcd1de43f64e55: Deleting consensus metadata
I20260926 06:53:03.156491 4826 catalog_manager.cc:4841] TS 75552adb4d054382b0fcd1de43f64e55 (127.0.191.195:44277): tablet a6fed4b381cc45b5903c4f1a19d7a106 (table test-table [id=168f3fc9e093475fbe1eb34dbaa462e2]) successfully deleted
W20260926 06:53:03.157538 4825 catalog_manager.cc:4584] TS 0bc51e45168e47ddbbff7aa798a4d94e (127.0.191.193:39787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6fed4b381cc45b5903c4f1a19d7a106: Network error: Client connection negotiation failed: client connection to 127.0.191.193:39787: connect: Connection refused (error 111)
I20260926 06:53:03.160614 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 5167
I20260926 06:53:03.165283 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 4808
2026-09-26T06:53:03Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2028 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T06:53:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:03Z Disabled control of system clock
I20260926 06:53:03.191478 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:41299
--webserver_interface=127.0.191.254
--webserver_port=0
--builtin_ntp_servers=127.0.191.212:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:41299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 06:53:03.273046 5342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:03.273253 5342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:03.273274 5342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:03.274417 5342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:03.274457 5342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:03.274470 5342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:03.274483 5342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:03.274534 5342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:03.275872 5342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:41299
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:41299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:03.276108 5342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:03.276381 5342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:03.281044 5350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:03.281052 5347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:03.281219 5342 server_base.cc:1034] running on GCE node
W20260926 06:53:03.281051 5348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:03.281579 5342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:03.281870 5342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:03.283041 5342 hybrid_clock.cc:648] HybridClock initialized: now 1790405583283022 us; error 52 us; skew 500 ppm
I20260926 06:53:03.284196 5342 webserver.cc:460] Webserver started at http://127.0.191.254:39645/ using document root <none> and password file <none>
I20260926 06:53:03.284421 5342 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:03.284488 5342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:03.284641 5342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:03.285720 5342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "7273bb19f4f640618a1f80445e95bd00"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "5dabe23fea442f9a60c7ce82a797a56c"
server_key_iv: "6c2c3b4fa02d3e7c568387ea04393bec"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.286063 5342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "7273bb19f4f640618a1f80445e95bd00"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "5dabe23fea442f9a60c7ce82a797a56c"
server_key_iv: "6c2c3b4fa02d3e7c568387ea04393bec"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.287271 5342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:53:03.287941 5356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:03.288098 5342 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:03.288209 5342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "7273bb19f4f640618a1f80445e95bd00"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "5dabe23fea442f9a60c7ce82a797a56c"
server_key_iv: "6c2c3b4fa02d3e7c568387ea04393bec"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.288283 5342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:03.291481 5342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:03.291736 5342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:03.291877 5342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:03.295544 5342 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:41299
I20260926 06:53:03.295571 5408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:41299 every 8 connection(s)
I20260926 06:53:03.295881 5342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:03.296496 5409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:03.299144 5409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00: Bootstrap starting.
I20260926 06:53:03.299707 5409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:03.300004 5409 log.cc:824] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:03.300621 5409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00: No bootstrap required, opened a new log
I20260926 06:53:03.302215 5409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7273bb19f4f640618a1f80445e95bd00" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 41299 } }
I20260926 06:53:03.302348 5409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:03.302369 5409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7273bb19f4f640618a1f80445e95bd00, State: Initialized, Role: FOLLOWER
I20260926 06:53:03.302531 5409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7273bb19f4f640618a1f80445e95bd00" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 41299 } }
I20260926 06:53:03.302588 5409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:03.302609 5409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:03.302675 5409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:03.303181 5409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7273bb19f4f640618a1f80445e95bd00" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 41299 } }
I20260926 06:53:03.303293 5409 leader_election.cc:302] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7273bb19f4f640618a1f80445e95bd00; no voters:
I20260926 06:53:03.303445 5409 leader_election.cc:288] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:03.303483 5412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:03.303653 5412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [term 1 LEADER]: Becoming Leader. State: Replica: 7273bb19f4f640618a1f80445e95bd00, State: Running, Role: LEADER
I20260926 06:53:03.303695 5409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:03.303758 5412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7273bb19f4f640618a1f80445e95bd00" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 41299 } }
I20260926 06:53:03.304087 5413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7273bb19f4f640618a1f80445e95bd00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7273bb19f4f640618a1f80445e95bd00" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 41299 } } }
I20260926 06:53:03.304183 5413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:03.304311 5414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7273bb19f4f640618a1f80445e95bd00. Latest consensus state: current_term: 1 leader_uuid: "7273bb19f4f640618a1f80445e95bd00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7273bb19f4f640618a1f80445e95bd00" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 41299 } } }
I20260926 06:53:03.304381 5414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:03.304430 5420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:03.304937 5420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:03.305264 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 5342
I20260926 06:53:03.305361 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:03.305580 767 external_mini_cluster.cc:1428] Setting key 7781c815c06e05b04aede4a88dbd8f46
I20260926 06:53:03.307296 5420 catalog_manager.cc:1347] Generated new cluster ID: dfce607afb814de0b600bfd71430f7b5
I20260926 06:53:03.307349 5420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:53:03.310127 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.306264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60510 (local address 127.0.191.254:41299)
0926 06:53:03.306442 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.306451 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.306485 (+ 34us) server_negotiation.cc:407] Connection header received
0926 06:53:03.306609 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.306614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.306646 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.306777 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:03.309656 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:03.309660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:03.309660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:03.309665 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:03.309713 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:03.309738 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:03.309821 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.309886 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.309944 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":77,"threads_started":1}
I20260926 06:53:03.318385 5420 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:03.318534 5420 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:03.327512 5420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00: Generated new TSK 0
I20260926 06:53:03.327718 5420 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:53:03.335605 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:41299
--builtin_ntp_servers=127.0.191.212:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 06:53:03.415766 5433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:03.415974 5433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:03.415997 5433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:03.417160 5433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:03.417204 5433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:03.417230 5433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:53:03.418629 5433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:41299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:03.418862 5433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:03.419166 5433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:03.424774 5438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:03.424780 5439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:03.425040 5441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:03.425143 5433 server_base.cc:1034] running on GCE node
I20260926 06:53:03.425334 5433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:03.425599 5433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:03.426749 5433 hybrid_clock.cc:648] HybridClock initialized: now 1790405583426742 us; error 32 us; skew 500 ppm
I20260926 06:53:03.427937 5433 webserver.cc:460] Webserver started at http://127.0.191.193:44995/ using document root <none> and password file <none>
I20260926 06:53:03.428182 5433 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:03.428227 5433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:03.428395 5433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:03.429499 5433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "b15bedb46a6c42cdaa9845077878e38b"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "fcc5b65bc29301b4534c8a47ce4fe1ae"
server_key_iv: "2e9e564a8158a1a374b4397afa5495f4"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.429847 5433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "b15bedb46a6c42cdaa9845077878e38b"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "fcc5b65bc29301b4534c8a47ce4fe1ae"
server_key_iv: "2e9e564a8158a1a374b4397afa5495f4"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.431097 5433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:03.431767 5447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:03.431945 5433 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:03.432031 5433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "b15bedb46a6c42cdaa9845077878e38b"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "fcc5b65bc29301b4534c8a47ce4fe1ae"
server_key_iv: "2e9e564a8158a1a374b4397afa5495f4"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.432101 5433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:03.435470 5433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:03.435740 5433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:03.435869 5433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:03.436087 5433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:03.436376 5433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:03.436407 5433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:03.436427 5433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:03.436470 5433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:03.442075 5433 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:43705
I20260926 06:53:03.442116 5560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:43705 every 8 connection(s)
I20260926 06:53:03.442407 5433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:53:03.444947 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.443046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:41233 (local address 127.0.191.254:41299)
0926 06:53:03.443104 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.443106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.443304 (+ 198us) server_negotiation.cc:407] Connection header received
0926 06:53:03.443706 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.443708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.443714 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.443766 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:03.444768 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:03.444770 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:03.444770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:03.444773 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:03.444786 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:03.444789 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:03.444834 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.444884 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.444896 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:53:03.445115 5563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.443025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:41299 (local address 127.0.191.193:41233)
0926 06:53:03.443191 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:03.443221 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:03.443620 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:03.443814 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:03.443817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:03.443824 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:03.444629 (+ 805us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:03.444629 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:03.444655 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:03.444693 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:03.444831 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:03.444833 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:03.444836 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:03.444885 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 06:53:03.444929 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":91,"threads_started":1}
I20260926 06:53:03.445605 5561 heartbeater.cc:347] Connected to a master server at 127.0.191.254:41299
I20260926 06:53:03.445703 5561 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:03.445850 5561 heartbeater.cc:511] Master 127.0.191.254:41299 requested a full tablet report, sending...
I20260926 06:53:03.446329 5373 ts_manager.cc:194] Registered new tserver with Master: b15bedb46a6c42cdaa9845077878e38b (127.0.191.193:43705)
I20260926 06:53:03.450323 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 5433
I20260926 06:53:03.450402 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:53:03.450534 767 external_mini_cluster.cc:1428] Setting key d6ef9c71e8b92b9e7966a06de465cb84
I20260926 06:53:03.451340 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:41299
--builtin_ntp_servers=127.0.191.212:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 06:53:03.529245 5564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:03.529526 5564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:03.529563 5564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:03.530776 5564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:03.530835 5564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:03.530879 5564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:53:03.532182 5564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:41299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:03.532428 5564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:03.532719 5564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:03.537631 5570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:03.537606 5569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:03.537580 5572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:03.537833 5564 server_base.cc:1034] running on GCE node
I20260926 06:53:03.538097 5564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:03.538333 5564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:03.539537 5564 hybrid_clock.cc:648] HybridClock initialized: now 1790405583539522 us; error 27 us; skew 500 ppm
I20260926 06:53:03.540822 5564 webserver.cc:460] Webserver started at http://127.0.191.194:36057/ using document root <none> and password file <none>
I20260926 06:53:03.541115 5564 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:03.541188 5564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:03.541320 5564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:03.542351 5564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "f38fd3b32a0d4b949e4ca5e1c473458d"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "c7fb9266e81914fa9c9e2c85bb638a91"
server_key_iv: "9e5f497d851357cb435cafd163895a35"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.542707 5564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "f38fd3b32a0d4b949e4ca5e1c473458d"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "c7fb9266e81914fa9c9e2c85bb638a91"
server_key_iv: "9e5f497d851357cb435cafd163895a35"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.543951 5564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:53:03.544634 5578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:03.544807 5564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:03.544901 5564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "f38fd3b32a0d4b949e4ca5e1c473458d"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "c7fb9266e81914fa9c9e2c85bb638a91"
server_key_iv: "9e5f497d851357cb435cafd163895a35"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.544988 5564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:03.548448 5564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:03.548694 5564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:03.548825 5564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:03.549034 5564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:03.549322 5564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:03.549374 5564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:03.549424 5564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:03.549453 5564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:03.554946 5564 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:45617
I20260926 06:53:03.554979 5691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:45617 every 8 connection(s)
I20260926 06:53:03.555305 5564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:53:03.558106 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.556049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:39783 (local address 127.0.191.254:41299)
0926 06:53:03.556110 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.556112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.556344 (+ 232us) server_negotiation.cc:407] Connection header received
0926 06:53:03.556757 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.556758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.556762 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.556817 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:03.557912 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:03.557914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:03.557914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:03.557917 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:03.557932 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:03.557936 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:03.557985 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.558043 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.558055 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:53:03.558303 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.556043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:41299 (local address 127.0.191.194:39783)
0926 06:53:03.556218 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:03.556255 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:03.556680 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:03.556867 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:03.556870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:03.556877 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:03.557781 (+ 904us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:03.557781 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:03.557808 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:03.557834 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:03.557981 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:03.557984 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:03.557986 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:53:03.558040 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 06:53:03.558084 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260926 06:53:03.558964 5692 heartbeater.cc:347] Connected to a master server at 127.0.191.254:41299
I20260926 06:53:03.559067 5692 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:03.559219 5692 heartbeater.cc:511] Master 127.0.191.254:41299 requested a full tablet report, sending...
I20260926 06:53:03.559558 5373 ts_manager.cc:194] Registered new tserver with Master: f38fd3b32a0d4b949e4ca5e1c473458d (127.0.191.194:45617)
I20260926 06:53:03.565491 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 5564
I20260926 06:53:03.565598 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:53:03.565802 767 external_mini_cluster.cc:1428] Setting key edd1b84cc2333ed0b6b406af9149a0bb
I20260926 06:53:03.566679 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:41299
--builtin_ntp_servers=127.0.191.212:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 06:53:03.645958 5695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:03.646147 5695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:03.646168 5695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:03.647236 5695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:03.647269 5695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:03.647293 5695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:53:03.648702 5695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:41299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:03.648880 5695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:03.649186 5695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:03.654385 5703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:03.654394 5701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:03.654587 5700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:03.654729 5695 server_base.cc:1034] running on GCE node
I20260926 06:53:03.654943 5695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:03.655177 5695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:03.656334 5695 hybrid_clock.cc:648] HybridClock initialized: now 1790405583656324 us; error 29 us; skew 500 ppm
I20260926 06:53:03.657684 5695 webserver.cc:460] Webserver started at http://127.0.191.195:34225/ using document root <none> and password file <none>
I20260926 06:53:03.657930 5695 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:03.657995 5695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:03.658141 5695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:03.659196 5695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "0a01c5db0e3643268526c5951a4a8593"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "30bb64a943570ec584899f00bfc8fe5f"
server_key_iv: "d52b21ca698375ce068ec14cecf337f7"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.659591 5695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "0a01c5db0e3643268526c5951a4a8593"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "30bb64a943570ec584899f00bfc8fe5f"
server_key_iv: "d52b21ca698375ce068ec14cecf337f7"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.660900 5695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:53:03.661685 5709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:03.661852 5695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:53:03.661947 5695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "0a01c5db0e3643268526c5951a4a8593"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "30bb64a943570ec584899f00bfc8fe5f"
server_key_iv: "d52b21ca698375ce068ec14cecf337f7"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.662027 5695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:03.665161 5695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:03.665494 5695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:03.665627 5695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:03.665836 5695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:03.666122 5695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:03.666174 5695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:03.666211 5695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:03.666246 5695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:03.671428 5695 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:34515
I20260926 06:53:03.671491 5822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:34515 every 8 connection(s)
I20260926 06:53:03.671804 5695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:53:03.674762 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.672521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:53903 (local address 127.0.191.254:41299)
0926 06:53:03.672581 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.672583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.672802 (+ 219us) server_negotiation.cc:407] Connection header received
0926 06:53:03.673284 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.673285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.673291 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.673363 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:03.674498 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:03.674500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:03.674500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:03.674504 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:03.674519 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:03.674524 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:03.674574 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.674676 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.674692 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 06:53:03.674960 5825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.672499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:41299 (local address 127.0.191.195:53903)
0926 06:53:03.672691 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:03.672721 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:03.673179 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:03.673460 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:03.673463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:03.673474 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:03.674344 (+ 870us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:03.674344 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:03.674370 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:03.674396 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:03.674577 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:03.674582 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:03.674584 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:53:03.674685 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 06:53:03.674744 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":77,"threads_started":1}
I20260926 06:53:03.675406 5823 heartbeater.cc:347] Connected to a master server at 127.0.191.254:41299
I20260926 06:53:03.675493 5823 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:03.675604 5823 heartbeater.cc:511] Master 127.0.191.254:41299 requested a full tablet report, sending...
I20260926 06:53:03.675933 5373 ts_manager.cc:194] Registered new tserver with Master: 0a01c5db0e3643268526c5951a4a8593 (127.0.191.195:34515)
I20260926 06:53:03.681164 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 5695
I20260926 06:53:03.681254 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:53:03.681430 767 external_mini_cluster.cc:1428] Setting key 1a914e83697d24efaea3b52a95e2d475
I20260926 06:53:03.682273 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:53:03.684536 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.683549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60522 (local address 127.0.191.254:41299)
0926 06:53:03.683598 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.683600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.683730 (+ 130us) server_negotiation.cc:407] Connection header received
0926 06:53:03.683766 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.683767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.683771 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.683810 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:03.684319 (+ 509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:03.684320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:03.684321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:03.684324 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:03.684338 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:03.684342 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:03.684383 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.684418 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.684429 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 06:53:03.686161 5373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:53:03.687562 5373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:53:03.691304 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.689551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:45617 (local address 127.0.0.1:50234)
0926 06:53:03.689712 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:03.689751 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:03.689805 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:03.690170 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:03.690175 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:03.690182 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:03.690859 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:03.690859 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:03.690864 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:03.690874 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:03.691095 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:03.691100 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:03.691103 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:03.691199 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 06:53:03.691211 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"spinlock_wait_cycles":768,"thread_start_us":103,"threads_started":1}
I20260926 06:53:03.691335 5835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.689837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50234 (local address 127.0.191.194:45617)
0926 06:53:03.690020 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.690025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.690036 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:53:03.690063 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.690065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.690074 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.690145 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:03.690966 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:03.690972 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:03.690973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:03.690978 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:03.691017 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:03.691041 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:03.691104 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.691214 (+ 110us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.691233 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
I20260926 06:53:03.691398 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.689759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:34515 (local address 127.0.0.1:57426)
0926 06:53:03.689974 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:03.689983 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:03.690022 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:03.690653 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:03.690654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:03.690656 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:03.691004 (+ 348us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:03.691005 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:03.691012 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:03.691023 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:03.691266 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:03.691267 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:03.691267 (+ 0us) client_negotiation.cc:769] Sending connection context
0926 06:53:03.691309 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 06:53:03.691321 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":62,"threads_started":1}
I20260926 06:53:03.691594 5836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.689936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57426 (local address 127.0.191.195:34515)
0926 06:53:03.690516 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.690519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.690525 (+ 6us) server_negotiation.cc:407] Connection header received
0926 06:53:03.690543 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.690545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.690550 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.690645 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:03.691109 (+ 464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:03.691114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:03.691115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:03.691144 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:03.691172 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:03.691190 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:03.691242 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.691473 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.691491 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":75,"threads_started":1}
I20260926 06:53:03.691982 5626 tablet_service.cc:1467] Processing CreateTablet for tablet 7eec6926dd3b4a7c8f26f98fe928ae1b (DEFAULT_TABLE table=test-table [id=c61f35ca54ac4cd4b9cdb79dd8694ba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:03.692312 5626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eec6926dd3b4a7c8f26f98fe928ae1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:03.692241 5757 tablet_service.cc:1467] Processing CreateTablet for tablet 7eec6926dd3b4a7c8f26f98fe928ae1b (DEFAULT_TABLE table=test-table [id=c61f35ca54ac4cd4b9cdb79dd8694ba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:03.692520 5757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eec6926dd3b4a7c8f26f98fe928ae1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:03.693691 5831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.689551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:43705 (local address 127.0.0.1:39684)
0926 06:53:03.689717 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:03.689750 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:03.689804 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:03.690226 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:03.690227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:03.690230 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:03.690723 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:03.690730 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:03.690747 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:03.690772 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:03.693581 (+ 2809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:03.693582 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:03.693583 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:53:03.693614 (+ 31us) client_negotiation.cc:240] Negotiation successful
0926 06:53:03.693628 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":4,"thread_start_us":53,"threads_started":1}
I20260926 06:53:03.694010 5833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.689645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39684 (local address 127.0.191.193:43705)
0926 06:53:03.690057 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.690061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.690071 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:53:03.690096 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.690098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.690107 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.690212 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:03.691566 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:03.691569 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:03.691569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:03.691573 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:03.693458 (+ 1885us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:03.693486 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:03.693867 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.693893 (+ 26us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.693909 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":90,"threads_started":1}
I20260926 06:53:03.695338 5838 tablet_bootstrap.cc:492] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593: Bootstrap starting.
I20260926 06:53:03.695493 5837 tablet_bootstrap.cc:492] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d: Bootstrap starting.
I20260926 06:53:03.695785 5495 tablet_service.cc:1467] Processing CreateTablet for tablet 7eec6926dd3b4a7c8f26f98fe928ae1b (DEFAULT_TABLE table=test-table [id=c61f35ca54ac4cd4b9cdb79dd8694ba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:03.695994 5838 tablet_bootstrap.cc:654] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:03.696099 5837 tablet_bootstrap.cc:654] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:03.696100 5495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eec6926dd3b4a7c8f26f98fe928ae1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:03.696296 5838 log.cc:824] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:03.696406 5837 log.cc:824] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:03.697101 5838 tablet_bootstrap.cc:492] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593: No bootstrap required, opened a new log
I20260926 06:53:03.697101 5837 tablet_bootstrap.cc:492] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d: No bootstrap required, opened a new log
I20260926 06:53:03.697173 5837 ts_tablet_manager.cc:1397] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:53:03.697283 5838 ts_tablet_manager.cc:1397] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:53:03.698820 5841 tablet_bootstrap.cc:492] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b: Bootstrap starting.
I20260926 06:53:03.699347 5841 tablet_bootstrap.cc:654] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:03.699635 5841 log.cc:824] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:03.699831 5837 raft_consensus.cc:348] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38fd3b32a0d4b949e4ca5e1c473458d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45617 } } peers { permanent_uuid: "b15bedb46a6c42cdaa9845077878e38b" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 43705 } } peers { permanent_uuid: "0a01c5db0e3643268526c5951a4a8593" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34515 } }
I20260926 06:53:03.699893 5838 raft_consensus.cc:348] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38fd3b32a0d4b949e4ca5e1c473458d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45617 } } peers { permanent_uuid: "b15bedb46a6c42cdaa9845077878e38b" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 43705 } } peers { permanent_uuid: "0a01c5db0e3643268526c5951a4a8593" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34515 } }
I20260926 06:53:03.700002 5837 raft_consensus.cc:374] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:03.700011 5838 raft_consensus.cc:374] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:03.700042 5838 raft_consensus.cc:729] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a01c5db0e3643268526c5951a4a8593, State: Initialized, Role: FOLLOWER
I20260926 06:53:03.700042 5837 raft_consensus.cc:729] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f38fd3b32a0d4b949e4ca5e1c473458d, State: Initialized, Role: FOLLOWER
I20260926 06:53:03.700165 5838 consensus_queue.cc:260] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38fd3b32a0d4b949e4ca5e1c473458d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45617 } } peers { permanent_uuid: "b15bedb46a6c42cdaa9845077878e38b" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 43705 } } peers { permanent_uuid: "0a01c5db0e3643268526c5951a4a8593" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34515 } }
I20260926 06:53:03.700199 5837 consensus_queue.cc:260] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38fd3b32a0d4b949e4ca5e1c473458d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45617 } } peers { permanent_uuid: "b15bedb46a6c42cdaa9845077878e38b" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 43705 } } peers { permanent_uuid: "0a01c5db0e3643268526c5951a4a8593" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34515 } }
I20260926 06:53:03.700350 5841 tablet_bootstrap.cc:492] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b: No bootstrap required, opened a new log
I20260926 06:53:03.700413 5841 ts_tablet_manager.cc:1397] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:53:03.700424 5838 ts_tablet_manager.cc:1428] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:53:03.700424 5837 ts_tablet_manager.cc:1428] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:53:03.700532 5823 heartbeater.cc:503] Master 127.0.191.254:41299 was elected leader, sending a full tablet report...
I20260926 06:53:03.700553 5692 heartbeater.cc:503] Master 127.0.191.254:41299 was elected leader, sending a full tablet report...
I20260926 06:53:03.702677 5841 raft_consensus.cc:348] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38fd3b32a0d4b949e4ca5e1c473458d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45617 } } peers { permanent_uuid: "b15bedb46a6c42cdaa9845077878e38b" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 43705 } } peers { permanent_uuid: "0a01c5db0e3643268526c5951a4a8593" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34515 } }
I20260926 06:53:03.702816 5841 raft_consensus.cc:374] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:03.702860 5841 raft_consensus.cc:729] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b15bedb46a6c42cdaa9845077878e38b, State: Initialized, Role: FOLLOWER
I20260926 06:53:03.702968 5841 consensus_queue.cc:260] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38fd3b32a0d4b949e4ca5e1c473458d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45617 } } peers { permanent_uuid: "b15bedb46a6c42cdaa9845077878e38b" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 43705 } } peers { permanent_uuid: "0a01c5db0e3643268526c5951a4a8593" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34515 } }
I20260926 06:53:03.703241 5841 ts_tablet_manager.cc:1428] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:53:03.703262 5561 heartbeater.cc:503] Master 127.0.191.254:41299 was elected leader, sending a full tablet report...
I20260926 06:53:03.706389 5845 raft_consensus.cc:482] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:53:03.706450 5845 raft_consensus.cc:504] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38fd3b32a0d4b949e4ca5e1c473458d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45617 } } peers { permanent_uuid: "b15bedb46a6c42cdaa9845077878e38b" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 43705 } } peers { permanent_uuid: "0a01c5db0e3643268526c5951a4a8593" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34515 } }
I20260926 06:53:03.706660 5845 leader_election.cc:288] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f38fd3b32a0d4b949e4ca5e1c473458d (127.0.191.194:45617), 0a01c5db0e3643268526c5951a4a8593 (127.0.191.195:34515)
I20260926 06:53:03.707911 5835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.706873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:38431 (local address 127.0.191.194:45617)
0926 06:53:03.706913 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.706916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.707077 (+ 161us) server_negotiation.cc:407] Connection header received
0926 06:53:03.707152 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.707153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.707158 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.707213 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:03.707707 (+ 494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:03.707709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:03.707709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:03.707714 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:03.707726 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:03.707731 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:03.707787 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.707840 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.707852 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:53:03.707911 5563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.706814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:45617 (local address 127.0.191.193:38431)
0926 06:53:03.707034 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:03.707044 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:03.707085 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:03.707217 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:03.707218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:03.707220 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:03.707623 (+ 403us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:03.707624 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:03.707629 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:03.707638 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:03.707786 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:03.707786 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:03.707787 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:53:03.707838 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 06:53:03.707851 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260926 06:53:03.708144 5836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.706987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:54711 (local address 127.0.191.195:34515)
0926 06:53:03.707039 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.707040 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.707112 (+ 72us) server_negotiation.cc:407] Connection header received
0926 06:53:03.707179 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.707180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.707184 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.707232 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:03.707911 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:03.707912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:03.707912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:03.707916 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:03.707929 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:03.707933 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:03.707983 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.708071 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.708086 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 06:53:03.708148 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.706921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:34515 (local address 127.0.191.193:54711)
0926 06:53:03.707059 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:03.707069 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:03.707123 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:03.707252 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:03.707253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:03.707256 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:03.707807 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:03.707808 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:03.707814 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:03.707826 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:03.708011 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:03.708012 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:03.708013 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:53:03.708069 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 06:53:03.708082 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260926 06:53:03.708226 5646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eec6926dd3b4a7c8f26f98fe928ae1b" candidate_uuid: "b15bedb46a6c42cdaa9845077878e38b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f38fd3b32a0d4b949e4ca5e1c473458d" is_pre_election: true
I20260926 06:53:03.708329 5646 raft_consensus.cc:2454] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b15bedb46a6c42cdaa9845077878e38b in term 0.
I20260926 06:53:03.708554 5451 leader_election.cc:302] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b15bedb46a6c42cdaa9845077878e38b, f38fd3b32a0d4b949e4ca5e1c473458d; no voters:
I20260926 06:53:03.708678 5845 raft_consensus.cc:2781] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:53:03.708670 5777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eec6926dd3b4a7c8f26f98fe928ae1b" candidate_uuid: "b15bedb46a6c42cdaa9845077878e38b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a01c5db0e3643268526c5951a4a8593" is_pre_election: true
I20260926 06:53:03.708724 5845 raft_consensus.cc:482] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:53:03.708747 5845 raft_consensus.cc:3037] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:03.708781 5777 raft_consensus.cc:2454] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b15bedb46a6c42cdaa9845077878e38b in term 0.
I20260926 06:53:03.709489 5845 raft_consensus.cc:504] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38fd3b32a0d4b949e4ca5e1c473458d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45617 } } peers { permanent_uuid: "b15bedb46a6c42cdaa9845077878e38b" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 43705 } } peers { permanent_uuid: "0a01c5db0e3643268526c5951a4a8593" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34515 } }
I20260926 06:53:03.709604 5845 leader_election.cc:288] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [CANDIDATE]: Term 1 election: Requested vote from peers f38fd3b32a0d4b949e4ca5e1c473458d (127.0.191.194:45617), 0a01c5db0e3643268526c5951a4a8593 (127.0.191.195:34515)
I20260926 06:53:03.709772 5777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eec6926dd3b4a7c8f26f98fe928ae1b" candidate_uuid: "b15bedb46a6c42cdaa9845077878e38b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a01c5db0e3643268526c5951a4a8593"
I20260926 06:53:03.709779 5646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eec6926dd3b4a7c8f26f98fe928ae1b" candidate_uuid: "b15bedb46a6c42cdaa9845077878e38b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f38fd3b32a0d4b949e4ca5e1c473458d"
I20260926 06:53:03.709863 5777 raft_consensus.cc:3037] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:03.709859 5646 raft_consensus.cc:3037] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:03.710489 5646 raft_consensus.cc:2454] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b15bedb46a6c42cdaa9845077878e38b in term 1.
I20260926 06:53:03.710508 5777 raft_consensus.cc:2454] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b15bedb46a6c42cdaa9845077878e38b in term 1.
I20260926 06:53:03.710655 5451 leader_election.cc:302] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b15bedb46a6c42cdaa9845077878e38b, f38fd3b32a0d4b949e4ca5e1c473458d; no voters:
I20260926 06:53:03.710739 5845 raft_consensus.cc:2781] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:03.710879 5845 raft_consensus.cc:686] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [term 1 LEADER]: Becoming Leader. State: Replica: b15bedb46a6c42cdaa9845077878e38b, State: Running, Role: LEADER
I20260926 06:53:03.710965 5845 consensus_queue.cc:237] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38fd3b32a0d4b949e4ca5e1c473458d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45617 } } peers { permanent_uuid: "b15bedb46a6c42cdaa9845077878e38b" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 43705 } } peers { permanent_uuid: "0a01c5db0e3643268526c5951a4a8593" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34515 } }
I20260926 06:53:03.711685 5371 catalog_manager.cc:5495] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b reported cstate change: term changed from 0 to 1, leader changed from <none> to b15bedb46a6c42cdaa9845077878e38b (127.0.191.193). New cstate: current_term: 1 leader_uuid: "b15bedb46a6c42cdaa9845077878e38b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38fd3b32a0d4b949e4ca5e1c473458d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b15bedb46a6c42cdaa9845077878e38b" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 43705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a01c5db0e3643268526c5951a4a8593" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34515 } health_report { overall_health: UNKNOWN } } }
I20260926 06:53:03.719730 5833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.718733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39692 (local address 127.0.191.193:43705)
0926 06:53:03.718779 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.718781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.718790 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:03.718872 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.718873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.718883 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.718953 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:03.719529 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:03.719531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:03.719531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:03.719536 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:03.719549 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:03.719552 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:03.719602 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.719654 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.719667 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 06:53:03.721302 5777 raft_consensus.cc:1261] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593 [term 1 FOLLOWER]: Refusing update from remote peer b15bedb46a6c42cdaa9845077878e38b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:03.721328 5646 raft_consensus.cc:1261] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d [term 1 FOLLOWER]: Refusing update from remote peer b15bedb46a6c42cdaa9845077878e38b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:03.721623 5845 consensus_queue.cc:1035] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a01c5db0e3643268526c5951a4a8593" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:03.721776 5847 consensus_queue.cc:1035] T 7eec6926dd3b4a7c8f26f98fe928ae1b P b15bedb46a6c42cdaa9845077878e38b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f38fd3b32a0d4b949e4ca5e1c473458d" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:03.724061 5852 mvcc.cc:205] Tried to move back new op lower bound from 7333501270920331264 to 7333501270880808960. Current Snapshot: MvccSnapshot[applied={T|T < 7333501270920331264}]
I20260926 06:53:03.724107 5851 mvcc.cc:205] Tried to move back new op lower bound from 7333501270920331264 to 7333501270880808960. Current Snapshot: MvccSnapshot[applied={T|T < 7333501270920331264}]
I20260926 06:53:03.726065 5372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:03.726197 5372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:03.726910 5372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7273bb19f4f640618a1f80445e95bd00: Sending DeleteTablet for 3 replicas of tablet 7eec6926dd3b4a7c8f26f98fe928ae1b
I20260926 06:53:03.727280 5626 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eec6926dd3b4a7c8f26f98fe928ae1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:03 UTC) from {username='slave'} at 127.0.0.1:50234
I20260926 06:53:03.727322 5757 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eec6926dd3b4a7c8f26f98fe928ae1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:03 UTC) from {username='slave'} at 127.0.0.1:57426
I20260926 06:53:03.727475 5860 tablet_replica.cc:330] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d: stopping tablet replica
I20260926 06:53:03.727560 5861 tablet_replica.cc:330] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593: stopping tablet replica
I20260926 06:53:03.727571 5860 raft_consensus.cc:2229] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:03.727659 5860 raft_consensus.cc:2258] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:03.727735 5861 raft_consensus.cc:2229] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:03.727783 5495 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eec6926dd3b4a7c8f26f98fe928ae1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:03 UTC) from {username='slave'} at 127.0.0.1:39684
I20260926 06:53:03.727908 5861 raft_consensus.cc:2258] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:03.727982 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 5433
I20260926 06:53:03.728334 5861 ts_tablet_manager.cc:1905] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:03.729977 5861 ts_tablet_manager.cc:1918] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:03.730048 5861 log.cc:1196] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/7eec6926dd3b4a7c8f26f98fe928ae1b
I20260926 06:53:03.730288 5861 ts_tablet_manager.cc:1939] T 7eec6926dd3b4a7c8f26f98fe928ae1b P 0a01c5db0e3643268526c5951a4a8593: Deleting consensus metadata
I20260926 06:53:03.730477 5860 ts_tablet_manager.cc:1905] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:03.730639 5360 catalog_manager.cc:4841] TS 0a01c5db0e3643268526c5951a4a8593 (127.0.191.195:34515): tablet 7eec6926dd3b4a7c8f26f98fe928ae1b (table test-table [id=c61f35ca54ac4cd4b9cdb79dd8694ba6]) successfully deleted
I20260926 06:53:03.731602 5860 ts_tablet_manager.cc:1918] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:03.731657 5860 log.cc:1196] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/7eec6926dd3b4a7c8f26f98fe928ae1b
I20260926 06:53:03.731896 5860 ts_tablet_manager.cc:1939] T 7eec6926dd3b4a7c8f26f98fe928ae1b P f38fd3b32a0d4b949e4ca5e1c473458d: Deleting consensus metadata
I20260926 06:53:03.732272 5360 catalog_manager.cc:4841] TS f38fd3b32a0d4b949e4ca5e1c473458d (127.0.191.194:45617): tablet 7eec6926dd3b4a7c8f26f98fe928ae1b (table test-table [id=c61f35ca54ac4cd4b9cdb79dd8694ba6]) successfully deleted
W20260926 06:53:03.732924 5359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:43705 (error 108)
I20260926 06:53:03.733126 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 5564
I20260926 06:53:03.733680 5831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.733277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:43705 (local address 127.0.0.1:39700)
0926 06:53:03.733606 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:03.733644 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:43705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311}
W20260926 06:53:03.733819 5359 catalog_manager.cc:4584] TS b15bedb46a6c42cdaa9845077878e38b (127.0.191.193:43705): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7eec6926dd3b4a7c8f26f98fe928ae1b: Network error: Client connection negotiation failed: client connection to 127.0.191.193:43705: connect: Connection refused (error 111)
I20260926 06:53:03.738526 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 5695
I20260926 06:53:03.742864 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 5342
2026-09-26T06:53:03Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (576 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 06:53:03.761893 767 mini_oidc.cc:152] Starting JWKS server
I20260926 06:53:03.762974 767 webserver.cc:460] Webserver started at https://127.0.0.1:45003/ using document root <none> and password file <none>
I20260926 06:53:03.763110 767 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 06:53:03.763293 767 webserver.cc:460] Webserver started at http://127.0.0.1:38249/ using document root <none> and password file <none>
2026-09-26T06:53:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:03Z Disabled control of system clock
I20260926 06:53:03.770648 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33529
--webserver_interface=127.0.191.254
--webserver_port=0
--builtin_ntp_servers=127.0.191.212:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:33529
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45003/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/testchainca.cert with env {}
W20260926 06:53:03.848417 5869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:03.848604 5869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:03.848626 5869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:03.849709 5869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:03.849743 5869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:03.849756 5869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:03.849767 5869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:03.849779 5869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 06:53:03.849792 5869 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 06:53:03.849803 5869 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45003/jwks.json
I20260926 06:53:03.851202 5869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:33529
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45003/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:03.851383 5869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:03.851532 5869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:53:03.856241 5869 server_base.cc:1034] running on GCE node
W20260926 06:53:03.856210 5877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:03.856213 5875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:03.856189 5874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:03.856551 5869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:03.856781 5869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:03.857947 5869 hybrid_clock.cc:648] HybridClock initialized: now 1790405583857933 us; error 25 us; skew 500 ppm
I20260926 06:53:03.859167 5869 webserver.cc:460] Webserver started at http://127.0.191.254:40017/ using document root <none> and password file <none>
I20260926 06:53:03.859419 5869 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:03.859490 5869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:03.859632 5869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:03.860626 5869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "8173f9b3ada84805aa33e458f9f58d00"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "2e528199c02a916b98e3eff0b83e72eb"
server_key_iv: "751499d5eb93bede385fb3d7cf0a2b58"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.860968 5869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "8173f9b3ada84805aa33e458f9f58d00"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "2e528199c02a916b98e3eff0b83e72eb"
server_key_iv: "751499d5eb93bede385fb3d7cf0a2b58"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.862278 5869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:53:03.862984 5883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:03.863175 5869 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:03.863286 5869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "8173f9b3ada84805aa33e458f9f58d00"
format_stamp: "Formatted at 2026-09-26 06:53:03 on dist-test-slave-ws74"
server_key: "2e528199c02a916b98e3eff0b83e72eb"
server_key_iv: "751499d5eb93bede385fb3d7cf0a2b58"
server_key_version: "encryptionkey@0"
I20260926 06:53:03.863358 5869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:03.873435 5869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:03.873750 5869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:03.873888 5869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:03.877735 5869 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:33529
I20260926 06:53:03.877779 5935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:33529 every 8 connection(s)
I20260926 06:53:03.878082 5869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:03.878918 5936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:03.881316 5936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00: Bootstrap starting.
I20260926 06:53:03.881915 5936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:03.882208 5936 log.cc:824] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:03.882896 5936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00: No bootstrap required, opened a new log
I20260926 06:53:03.884565 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 5869
I20260926 06:53:03.884673 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:03.884560 5936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173f9b3ada84805aa33e458f9f58d00" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33529 } }
I20260926 06:53:03.884744 5936 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:03.884789 5936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8173f9b3ada84805aa33e458f9f58d00, State: Initialized, Role: FOLLOWER
I20260926 06:53:03.884857 767 external_mini_cluster.cc:1428] Setting key 0478abb3ea00bb41b2c9c5da921458c1
I20260926 06:53:03.884929 5936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173f9b3ada84805aa33e458f9f58d00" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33529 } }
I20260926 06:53:03.885011 5936 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:03.885069 5936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:03.885118 5936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:03.885771 5936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173f9b3ada84805aa33e458f9f58d00" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33529 } }
I20260926 06:53:03.885856 5936 leader_election.cc:302] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8173f9b3ada84805aa33e458f9f58d00; no voters:
I20260926 06:53:03.886044 5936 leader_election.cc:288] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:03.886068 5941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:03.886176 5941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [term 1 LEADER]: Becoming Leader. State: Replica: 8173f9b3ada84805aa33e458f9f58d00, State: Running, Role: LEADER
I20260926 06:53:03.886258 5941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173f9b3ada84805aa33e458f9f58d00" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33529 } }
I20260926 06:53:03.886313 5936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:03.886561 5941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8173f9b3ada84805aa33e458f9f58d00. Latest consensus state: current_term: 1 leader_uuid: "8173f9b3ada84805aa33e458f9f58d00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173f9b3ada84805aa33e458f9f58d00" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33529 } } }
I20260926 06:53:03.886624 5941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:03.886586 5942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8173f9b3ada84805aa33e458f9f58d00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173f9b3ada84805aa33e458f9f58d00" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33529 } } }
I20260926 06:53:03.886677 5942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:03.886823 5946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:03.887317 5946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:03.889089 5946 catalog_manager.cc:1347] Generated new cluster ID: cd1affad72aa4158abf7b9e1df578081
I20260926 06:53:03.889165 5946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:53:03.891268 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.885344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47018 (local address 127.0.191.254:33529)
0926 06:53:03.885564 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.885575 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.885606 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:53:03.885704 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.885709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.885770 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.885911 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:03.886161 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.887949 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.889060 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.889678 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.889803 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:03.890568 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:03.890572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:03.890572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:03.890785 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:03.890826 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:03.890840 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:03.890914 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:03.890989 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 06:53:03.891064 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260926 06:53:03.892174 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:53:03.898521 5946 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:03.899148 5946 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:03.904063 5946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8173f9b3ada84805aa33e458f9f58d00: Generated new TSK 0
I20260926 06:53:03.904227 5946 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 06:53:03.931074 5966 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 06:53:03.931196 5966 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 06:53:03.931406 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:03.915289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47032 (local address 127.0.191.254:33529)
0926 06:53:03.915357 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:03.915360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:03.915519 (+ 159us) server_negotiation.cc:407] Connection header received
0926 06:53:03.915582 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:03.915583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:03.915595 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:03.915673 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0926 06:53:03.915812 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.916571 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.917356 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.917694 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.917757 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:03.917945 (+ 188us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 06:53:03.931202 (+ 13257us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 06:53:03.931335 (+ 133us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47032: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":44}
W20260926 06:53:03.931464 5940 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47032: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 06:53:03.931819 5965 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:03.915224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33529 (local address 127.0.0.1:47032)
0926 06:53:03.915441 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:03.915450 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:03.915506 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:03.915686 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:03.915688 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:03.915689 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0926 06:53:03.915745 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:03.915747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.916656 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.916657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:03.917266 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:03.917268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:03.917776 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:03.917776 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:03.917828 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:03.917845 (+ 17us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 06:53:03.931387 (+ 13542us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 06:53:03.931503 (+ 116us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.191.254:33529: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":104,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.0.191.254:33529: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 06:53:03.932312 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 5869
2026-09-26T06:53:03Z chronyd exiting
I20260926 06:53:03.967873 767 test_util.cc:175] -----------------------------------------------
I20260926 06:53:03.967959 767 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790405559478510-767-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (207 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 06:53:03.968689 767 mini_oidc.cc:152] Starting JWKS server
I20260926 06:53:03.969362 767 webserver.cc:460] Webserver started at https://127.0.0.1:36041/ using document root <none> and password file <none>
I20260926 06:53:03.969478 767 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 06:53:03.969650 767 webserver.cc:460] Webserver started at http://127.0.0.1:35259/ using document root <none> and password file <none>
2026-09-26T06:53:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:03Z Disabled control of system clock
I20260926 06:53:03.976660 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33649
--webserver_interface=127.0.191.254
--webserver_port=0
--builtin_ntp_servers=127.0.191.212:46379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:33649
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36041/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/testchainca.cert with env {}
W20260926 06:53:04.053612 5972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:04.053833 5972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:04.053867 5972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:04.055017 5972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:04.055083 5972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:04.055104 5972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:04.055122 5972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:04.055150 5972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 06:53:04.055176 5972 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 06:53:04.055198 5972 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36041/jwks.json
I20260926 06:53:04.056634 5972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:33649
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36041/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:04.056903 5972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:04.057117 5972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:04.061818 5980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:04.061780 5978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:04.061930 5972 server_base.cc:1034] running on GCE node
W20260926 06:53:04.061760 5977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:04.062305 5972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:04.062636 5972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:04.063799 5972 hybrid_clock.cc:648] HybridClock initialized: now 1790405584063794 us; error 35 us; skew 500 ppm
I20260926 06:53:04.064926 5972 webserver.cc:460] Webserver started at http://127.0.191.254:42003/ using document root <none> and password file <none>
I20260926 06:53:04.065183 5972 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:04.065253 5972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:04.065418 5972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:04.066532 5972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "849ea11c6f0043bb9cae2bc7c1cf7aca"
format_stamp: "Formatted at 2026-09-26 06:53:04 on dist-test-slave-ws74"
server_key: "0daba44fd7eb3b301b2b6f3af8290cff"
server_key_iv: "cd42307962676b9354c15a532b2a4a71"
server_key_version: "encryptionkey@0"
I20260926 06:53:04.066885 5972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "849ea11c6f0043bb9cae2bc7c1cf7aca"
format_stamp: "Formatted at 2026-09-26 06:53:04 on dist-test-slave-ws74"
server_key: "0daba44fd7eb3b301b2b6f3af8290cff"
server_key_iv: "cd42307962676b9354c15a532b2a4a71"
server_key_version: "encryptionkey@0"
I20260926 06:53:04.068086 5972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:53:04.068749 5986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:04.068914 5972 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:53:04.069007 5972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "849ea11c6f0043bb9cae2bc7c1cf7aca"
format_stamp: "Formatted at 2026-09-26 06:53:04 on dist-test-slave-ws74"
server_key: "0daba44fd7eb3b301b2b6f3af8290cff"
server_key_iv: "cd42307962676b9354c15a532b2a4a71"
server_key_version: "encryptionkey@0"
I20260926 06:53:04.069101 5972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:04.100289 5972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:04.100611 5972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:04.100764 5972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:04.104578 5972 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:33649
I20260926 06:53:04.104614 6038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:33649 every 8 connection(s)
I20260926 06:53:04.104954 5972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:04.105906 6039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:04.108043 6039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca: Bootstrap starting.
I20260926 06:53:04.108593 6039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:04.108878 6039 log.cc:824] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:04.109556 6039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca: No bootstrap required, opened a new log
I20260926 06:53:04.111214 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 5972
I20260926 06:53:04.111114 6039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849ea11c6f0043bb9cae2bc7c1cf7aca" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33649 } }
I20260926 06:53:04.111282 6039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:04.111328 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:04.111322 6039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 849ea11c6f0043bb9cae2bc7c1cf7aca, State: Initialized, Role: FOLLOWER
I20260926 06:53:04.111467 6039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849ea11c6f0043bb9cae2bc7c1cf7aca" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33649 } }
I20260926 06:53:04.111518 767 external_mini_cluster.cc:1428] Setting key 27818e65fdc1111a31014510d20326d5
I20260926 06:53:04.111549 6039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:04.111584 6039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:04.111634 6039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:04.112293 6039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849ea11c6f0043bb9cae2bc7c1cf7aca" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33649 } }
I20260926 06:53:04.112375 6039 leader_election.cc:302] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 849ea11c6f0043bb9cae2bc7c1cf7aca; no voters:
I20260926 06:53:04.112587 6039 leader_election.cc:288] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:04.112613 6044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:04.112743 6044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [term 1 LEADER]: Becoming Leader. State: Replica: 849ea11c6f0043bb9cae2bc7c1cf7aca, State: Running, Role: LEADER
I20260926 06:53:04.112849 6039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:04.112845 6044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849ea11c6f0043bb9cae2bc7c1cf7aca" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33649 } }
I20260926 06:53:04.113492 6045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "849ea11c6f0043bb9cae2bc7c1cf7aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849ea11c6f0043bb9cae2bc7c1cf7aca" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33649 } } }
I20260926 06:53:04.113574 6045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:04.113488 6044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 849ea11c6f0043bb9cae2bc7c1cf7aca. Latest consensus state: current_term: 1 leader_uuid: "849ea11c6f0043bb9cae2bc7c1cf7aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849ea11c6f0043bb9cae2bc7c1cf7aca" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33649 } } }
I20260926 06:53:04.113758 6044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:04.113766 6053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:04.114403 6053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:04.115850 6053 catalog_manager.cc:1347] Generated new cluster ID: d6a83b426d484efdb86c4b7bdb1ff368
I20260926 06:53:04.115900 6053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:53:04.117653 6043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:04.111971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54342 (local address 127.0.191.254:33649)
0926 06:53:04.112195 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:04.112204 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:04.112233 (+ 29us) server_negotiation.cc:407] Connection header received
0926 06:53:04.112330 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:04.112335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:04.112431 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:04.112582 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:04.112860 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.114225 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.115588 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.116155 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.116276 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:04.116959 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:04.116963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:04.116964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:04.117151 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:04.117187 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:04.117199 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:04.117266 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:04.117362 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 06:53:04.117448 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260926 06:53:04.118461 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:53:04.129129 6053 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:04.129683 6053 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:04.135684 6053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 849ea11c6f0043bb9cae2bc7c1cf7aca: Generated new TSK 0
I20260926 06:53:04.135838 6053 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 06:53:04.155889 6069 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 06:53:04.155956 6069 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 06:53:04.156250 6043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:04.142152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54356 (local address 127.0.191.254:33649)
0926 06:53:04.142209 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:04.142211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:04.142295 (+ 84us) server_negotiation.cc:407] Connection header received
0926 06:53:04.142341 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:04.142342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:04.142354 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:04.142428 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0926 06:53:04.142534 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.143306 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.144026 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.144331 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.144398 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:04.144583 (+ 185us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 06:53:04.156086 (+ 11503us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36041/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 06:53:04.156200 (+ 114us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54356: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36041/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":35}
W20260926 06:53:04.156319 6043 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54356: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36041/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260926 06:53:04.156410 6068 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:04.142068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33649 (local address 127.0.0.1:54356)
0926 06:53:04.142225 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:04.142235 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:04.142292 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:04.142431 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:04.142431 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:04.142433 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 06:53:04.142478 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:04.142480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.143386 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.143386 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:04.143952 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:04.143954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.144403 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.144404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:04.144442 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:04.144446 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 06:53:04.156199 (+ 11753us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36041/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 06:53:04.156313 (+ 114us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.191.254:33649: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36041/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260926 06:53:04.156699 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 5972
2026-09-26T06:53:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (224 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 06:53:04.193240 767 mini_oidc.cc:152] Starting JWKS server
I20260926 06:53:04.193970 767 webserver.cc:460] Webserver started at https://127.0.0.1:39745/ using document root <none> and password file <none>
I20260926 06:53:04.194038 767 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 06:53:04.194216 767 webserver.cc:460] Webserver started at http://127.0.0.1:40851/ using document root <none> and password file <none>
2026-09-26T06:53:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:04Z Disabled control of system clock
I20260926 06:53:04.201573 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33111
--webserver_interface=127.0.191.254
--webserver_port=0
--builtin_ntp_servers=127.0.191.212:40319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:33111
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39745/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 06:53:04.279280 6075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:04.279457 6075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:04.279479 6075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:04.280540 6075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:04.280571 6075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:04.280584 6075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:04.280596 6075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:04.280608 6075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 06:53:04.280622 6075 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 06:53:04.280632 6075 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39745/jwks.json
I20260926 06:53:04.281895 6075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:40319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:33111
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39745/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:04.282064 6075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:04.282300 6075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:04.286944 6081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:04.286934 6083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:04.286975 6080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:04.286974 6075 server_base.cc:1034] running on GCE node
I20260926 06:53:04.287348 6075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:04.287609 6075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:04.288800 6075 hybrid_clock.cc:648] HybridClock initialized: now 1790405584288789 us; error 28 us; skew 500 ppm
I20260926 06:53:04.290014 6075 webserver.cc:460] Webserver started at http://127.0.191.254:41329/ using document root <none> and password file <none>
I20260926 06:53:04.290253 6075 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:04.290321 6075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:04.290469 6075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:04.291473 6075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "afc609e82a4840fe864a60d91da298fa"
format_stamp: "Formatted at 2026-09-26 06:53:04 on dist-test-slave-ws74"
server_key: "5e31606d00484723019453593645d372"
server_key_iv: "98718313a9919b4b6d46e250bc543254"
server_key_version: "encryptionkey@0"
I20260926 06:53:04.291821 6075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "afc609e82a4840fe864a60d91da298fa"
format_stamp: "Formatted at 2026-09-26 06:53:04 on dist-test-slave-ws74"
server_key: "5e31606d00484723019453593645d372"
server_key_iv: "98718313a9919b4b6d46e250bc543254"
server_key_version: "encryptionkey@0"
I20260926 06:53:04.293036 6075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:53:04.293821 6089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:04.293996 6075 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:04.294092 6075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "afc609e82a4840fe864a60d91da298fa"
format_stamp: "Formatted at 2026-09-26 06:53:04 on dist-test-slave-ws74"
server_key: "5e31606d00484723019453593645d372"
server_key_iv: "98718313a9919b4b6d46e250bc543254"
server_key_version: "encryptionkey@0"
I20260926 06:53:04.294180 6075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:04.316237 6075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:04.316550 6075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:04.316704 6075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:04.320499 6075 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:33111
I20260926 06:53:04.320536 6141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:33111 every 8 connection(s)
I20260926 06:53:04.320897 6075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:04.321573 6142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:04.324067 6142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa: Bootstrap starting.
I20260926 06:53:04.324633 6142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:04.324909 6142 log.cc:824] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:04.325493 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 6075
I20260926 06:53:04.325585 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:04.325678 6142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa: No bootstrap required, opened a new log
I20260926 06:53:04.325778 767 external_mini_cluster.cc:1428] Setting key 741b4a472a626d092bbe79731c6ff958
I20260926 06:53:04.327432 6142 raft_consensus.cc:348] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afc609e82a4840fe864a60d91da298fa" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33111 } }
I20260926 06:53:04.327584 6142 raft_consensus.cc:374] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:04.327628 6142 raft_consensus.cc:729] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afc609e82a4840fe864a60d91da298fa, State: Initialized, Role: FOLLOWER
I20260926 06:53:04.327756 6142 consensus_queue.cc:260] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afc609e82a4840fe864a60d91da298fa" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33111 } }
I20260926 06:53:04.327852 6142 raft_consensus.cc:388] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:04.327895 6142 raft_consensus.cc:482] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:04.327946 6142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:04.328627 6142 raft_consensus.cc:504] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afc609e82a4840fe864a60d91da298fa" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33111 } }
I20260926 06:53:04.328743 6142 leader_election.cc:302] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afc609e82a4840fe864a60d91da298fa; no voters:
I20260926 06:53:04.328933 6142 leader_election.cc:288] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:04.328963 6147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:04.329074 6147 raft_consensus.cc:686] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [term 1 LEADER]: Becoming Leader. State: Replica: afc609e82a4840fe864a60d91da298fa, State: Running, Role: LEADER
I20260926 06:53:04.329152 6147 consensus_queue.cc:237] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afc609e82a4840fe864a60d91da298fa" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33111 } }
I20260926 06:53:04.329180 6142 sys_catalog.cc:564] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:04.329507 6147 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader afc609e82a4840fe864a60d91da298fa. Latest consensus state: current_term: 1 leader_uuid: "afc609e82a4840fe864a60d91da298fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afc609e82a4840fe864a60d91da298fa" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33111 } } }
I20260926 06:53:04.329576 6147 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:04.329532 6149 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afc609e82a4840fe864a60d91da298fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afc609e82a4840fe864a60d91da298fa" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33111 } } }
I20260926 06:53:04.329717 6149 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:04.329794 6152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:04.330312 6152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:04.331853 6164 mvcc.cc:205] Tried to move back new op lower bound from 7333501273420800000 to 7333501273412694016. Current Snapshot: MvccSnapshot[applied={T|T < 7333501273420800000}]
I20260926 06:53:04.332252 6152 catalog_manager.cc:1347] Generated new cluster ID: d7923d0eca2a4824bf6ba243d129cfcd
I20260926 06:53:04.332309 6152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:53:04.332458 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:04.326256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41370 (local address 127.0.191.254:33111)
0926 06:53:04.326457 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:04.326465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:04.326495 (+ 30us) server_negotiation.cc:407] Connection header received
0926 06:53:04.327418 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:04.327424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:04.327480 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:04.327622 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:04.327823 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.328822 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.329896 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.330467 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.330567 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:04.331654 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:04.331659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:04.331659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:04.331865 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:04.331905 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:04.331921 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:04.331964 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:04.332122 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 06:53:04.332198 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260926 06:53:04.333215 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:53:04.343503 6152 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:04.343983 6152 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:04.355151 6152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afc609e82a4840fe864a60d91da298fa: Generated new TSK 0
I20260926 06:53:04.355335 6152 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:53:04.365497 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:04.361913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41382 (local address 127.0.191.254:33111)
0926 06:53:04.361975 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:04.361977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:04.362067 (+ 90us) server_negotiation.cc:407] Connection header received
0926 06:53:04.362102 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:04.362102 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:04.362114 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:04.362193 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0926 06:53:04.362317 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.363095 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.363855 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.364153 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.364222 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:04.364370 (+ 148us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 06:53:04.365277 (+ 907us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39745/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 06:53:04.365432 (+ 155us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41382: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39745/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":38}
W20260926 06:53:04.365551 6146 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41382: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39745/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 06:53:04.367419 6171 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:04.361831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33111 (local address 127.0.0.1:41382)
0926 06:53:04.361987 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:04.361997 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:04.362057 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:04.362196 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:04.362196 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:04.362198 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 06:53:04.362249 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:04.362251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.363176 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.363176 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:04.363771 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:04.363773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.364231 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.364231 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:04.364269 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:04.364273 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 06:53:04.365400 (+ 1127us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39745/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 06:53:04.367328 (+ 1928us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.191.254:33111: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39745/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":1767,"thread_start_us":95,"threads_started":1}
I20260926 06:53:04.367690 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 6075
E20260926 06:53:04.376631 6172 webserver.cc:533] Webserver: sslize failed:
E20260926 06:53:04.376694 6172 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-26T06:53:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (211 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T06:53:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:04Z Disabled control of system clock
I20260926 06:53:04.411336 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:42465
--webserver_interface=127.0.191.254
--webserver_port=0
--builtin_ntp_servers=127.0.191.212:39737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:42465
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/testchainca.cert with env {}
W20260926 06:53:04.488543 6176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:04.488731 6176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:04.488754 6176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:04.489833 6176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:04.489866 6176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:04.489878 6176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:04.489892 6176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:04.489902 6176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 06:53:04.489914 6176 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 06:53:04.489926 6176 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 06:53:04.491294 6176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:42465
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:42465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:04.491466 6176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:04.491603 6176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:04.496109 6181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:04.496133 6182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:04.496130 6184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:04.496201 6176 server_base.cc:1034] running on GCE node
I20260926 06:53:04.496529 6176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:04.496796 6176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:04.497928 6176 hybrid_clock.cc:648] HybridClock initialized: now 1790405584497914 us; error 25 us; skew 500 ppm
I20260926 06:53:04.499177 6176 webserver.cc:460] Webserver started at http://127.0.191.254:36521/ using document root <none> and password file <none>
I20260926 06:53:04.499413 6176 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:04.499457 6176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:04.499614 6176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:04.500571 6176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "29f9c0f8607e4ac480db0ee23457fb53"
format_stamp: "Formatted at 2026-09-26 06:53:04 on dist-test-slave-ws74"
server_key: "53cebfb54b314806a1a9a422747ab3d2"
server_key_iv: "6f072c30b3c1ca74b43a7a8961675697"
server_key_version: "encryptionkey@0"
I20260926 06:53:04.500912 6176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "29f9c0f8607e4ac480db0ee23457fb53"
format_stamp: "Formatted at 2026-09-26 06:53:04 on dist-test-slave-ws74"
server_key: "53cebfb54b314806a1a9a422747ab3d2"
server_key_iv: "6f072c30b3c1ca74b43a7a8961675697"
server_key_version: "encryptionkey@0"
I20260926 06:53:04.502197 6176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:53:04.502859 6190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:04.503031 6176 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:53:04.503120 6176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "29f9c0f8607e4ac480db0ee23457fb53"
format_stamp: "Formatted at 2026-09-26 06:53:04 on dist-test-slave-ws74"
server_key: "53cebfb54b314806a1a9a422747ab3d2"
server_key_iv: "6f072c30b3c1ca74b43a7a8961675697"
server_key_version: "encryptionkey@0"
I20260926 06:53:04.503177 6176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:04.512339 6176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:04.512539 6176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:04.512629 6176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:04.516171 6176 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:42465
I20260926 06:53:04.516234 6242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:42465 every 8 connection(s)
I20260926 06:53:04.516502 6176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:04.517446 6243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:04.519583 6243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53: Bootstrap starting.
I20260926 06:53:04.520136 6243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:04.520413 6243 log.cc:824] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:04.521063 6243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53: No bootstrap required, opened a new log
I20260926 06:53:04.522688 6243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29f9c0f8607e4ac480db0ee23457fb53" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42465 } }
I20260926 06:53:04.522838 6243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:04.522881 6243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29f9c0f8607e4ac480db0ee23457fb53, State: Initialized, Role: FOLLOWER
I20260926 06:53:04.523051 6243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29f9c0f8607e4ac480db0ee23457fb53" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42465 } }
I20260926 06:53:04.523135 6243 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:04.523175 6243 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:04.523223 6243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:04.523728 6243 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29f9c0f8607e4ac480db0ee23457fb53" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42465 } }
I20260926 06:53:04.523842 6243 leader_election.cc:302] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29f9c0f8607e4ac480db0ee23457fb53; no voters:
I20260926 06:53:04.524012 6243 leader_election.cc:288] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:04.524053 6246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:04.524164 6246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [term 1 LEADER]: Becoming Leader. State: Replica: 29f9c0f8607e4ac480db0ee23457fb53, State: Running, Role: LEADER
I20260926 06:53:04.524251 6246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29f9c0f8607e4ac480db0ee23457fb53" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42465 } }
I20260926 06:53:04.524277 6243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:04.524617 6248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29f9c0f8607e4ac480db0ee23457fb53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29f9c0f8607e4ac480db0ee23457fb53" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42465 } } }
I20260926 06:53:04.524706 6248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:04.524730 6247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29f9c0f8607e4ac480db0ee23457fb53. Latest consensus state: current_term: 1 leader_uuid: "29f9c0f8607e4ac480db0ee23457fb53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29f9c0f8607e4ac480db0ee23457fb53" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42465 } } }
I20260926 06:53:04.524825 6247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:04.525046 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 6176
I20260926 06:53:04.525144 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:04.525321 767 external_mini_cluster.cc:1428] Setting key 79e4959f611b622c8b838e085e5099f8
I20260926 06:53:04.526691 6252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:04.527189 6252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:04.529007 6252 catalog_manager.cc:1347] Generated new cluster ID: e29b0634b09a435a9cf8dace061a44ea
I20260926 06:53:04.529067 6252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:53:04.531157 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:04.525847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42766 (local address 127.0.191.254:42465)
0926 06:53:04.526037 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:04.526044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:04.526146 (+ 102us) server_negotiation.cc:407] Connection header received
0926 06:53:04.526275 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:04.526281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:04.526326 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:04.526426 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:04.526643 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.528035 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.529053 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.529666 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.529794 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:04.530473 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:04.530477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:04.530477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:04.530673 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:04.530721 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:04.530745 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:04.530813 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:04.530880 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 06:53:04.530954 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260926 06:53:04.532109 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:53:04.534930 6252 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:04.535470 6252 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:04.541904 6252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29f9c0f8607e4ac480db0ee23457fb53: Generated new TSK 0
I20260926 06:53:04.542073 6252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:53:04.592324 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:04.548524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42774 (local address 127.0.191.254:42465)
0926 06:53:04.548565 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:04.548567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:04.548665 (+ 98us) server_negotiation.cc:407] Connection header received
0926 06:53:04.548729 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:04.548729 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:04.548738 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:04.548788 (+ 50us) server_negotiation.cc:226] Negotiated authn=JWT
0926 06:53:04.548908 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.549429 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.550501 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.550967 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.551014 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:04.551192 (+ 178us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 06:53:04.592106 (+ 40914us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 06:53:04.592254 (+ 148us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42774: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":24}
W20260926 06:53:04.592408 6257 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42774: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 06:53:04.592509 6272 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:04.548442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:42465 (local address 127.0.0.1:42774)
0926 06:53:04.548594 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:04.548604 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:04.548667 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:04.548797 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:04.548797 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:04.548799 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 06:53:04.548853 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:04.548855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.549520 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.549520 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:04.550432 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:04.550435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:04.551044 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:04.551045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:04.551103 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:04.551107 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 06:53:04.592244 (+ 41137us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 06:53:04.592398 (+ 154us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.191.254:42465: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260926 06:53:04.592803 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 6176
2026-09-26T06:53:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (205 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (84 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (85 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:53:04 dist-test-slave-ws74 krb5kdc[6284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:53:04 dist-test-slave-ws74 krb5kdc[6284](info): set up 2 sockets
Sep 26 06:53:04 dist-test-slave-ws74 krb5kdc[6284](info): commencing operation
krb5kdc: starting...
W20260926 06:53:05.789525 767 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:53:05 dist-test-slave-ws74 krb5kdc[6284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:53:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:53:05.906438 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:42353
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:39511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:42353
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:05.988814 6297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:05.989007 6297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:05.989028 6297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:05.990356 6297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:05.990396 6297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:05.990410 6297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:05.990422 6297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:05.990434 6297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:05.991959 6297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:42353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:42353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:05.992172 6297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:05.992367 6297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:05.997184 6305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:05.997210 6302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:05.997494 6297 server_base.cc:1034] running on GCE node
W20260926 06:53:05.997236 6303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:05.997779 6297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:05.998059 6297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:05.999209 6297 hybrid_clock.cc:648] HybridClock initialized: now 1790405585999188 us; error 47 us; skew 500 ppm
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:06.009244 6297 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:53:06.009716 6297 webserver.cc:460] Webserver started at http://127.0.191.254:39743/ using document root <none> and password file <none>
I20260926 06:53:06.009898 6297 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:06.009938 6297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:06.010026 6297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:06.011023 6297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "93675bbb507d4074821057857921a858"
format_stamp: "Formatted at 2026-09-26 06:53:06 on dist-test-slave-ws74"
server_key: "c49dc4426ae9451b97146660f1f55374"
server_key_iv: "55baebd37c98e27a3bef5d767aa7e888"
server_key_version: "encryptionkey@0"
I20260926 06:53:06.011315 6297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "93675bbb507d4074821057857921a858"
format_stamp: "Formatted at 2026-09-26 06:53:06 on dist-test-slave-ws74"
server_key: "c49dc4426ae9451b97146660f1f55374"
server_key_iv: "55baebd37c98e27a3bef5d767aa7e888"
server_key_version: "encryptionkey@0"
I20260926 06:53:06.012521 6297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:53:06.013180 6312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:06.013389 6297 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:53:06.013470 6297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "93675bbb507d4074821057857921a858"
format_stamp: "Formatted at 2026-09-26 06:53:06 on dist-test-slave-ws74"
server_key: "c49dc4426ae9451b97146660f1f55374"
server_key_iv: "55baebd37c98e27a3bef5d767aa7e888"
server_key_version: "encryptionkey@0"
I20260926 06:53:06.013520 6297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:06.051391 6297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:06.053779 6297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:06.053891 6297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:06.057650 6297 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:42353
I20260926 06:53:06.057729 6364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:42353 every 8 connection(s)
I20260926 06:53:06.058236 6297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:06.058925 6365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:06.061116 6365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858: Bootstrap starting.
I20260926 06:53:06.061134 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 6297
I20260926 06:53:06.061254 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:06.061491 767 external_mini_cluster.cc:1428] Setting key eeb7ee6840c36f31bd3e4c4adbdf795e
I20260926 06:53:06.061851 6365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:06.062175 6365 log.cc:824] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:06.062885 6365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858: No bootstrap required, opened a new log
I20260926 06:53:06.064687 6365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93675bbb507d4074821057857921a858" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42353 } }
I20260926 06:53:06.064818 6365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:06.064841 6365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93675bbb507d4074821057857921a858, State: Initialized, Role: FOLLOWER
I20260926 06:53:06.064978 6365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93675bbb507d4074821057857921a858" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42353 } }
I20260926 06:53:06.065045 6365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:06.065099 6365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:06.065151 6365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:06.065755 6365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93675bbb507d4074821057857921a858" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42353 } }
I20260926 06:53:06.065871 6365 leader_election.cc:302] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93675bbb507d4074821057857921a858; no voters:
I20260926 06:53:06.066056 6365 leader_election.cc:288] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:06.066084 6370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:06.066197 6370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [term 1 LEADER]: Becoming Leader. State: Replica: 93675bbb507d4074821057857921a858, State: Running, Role: LEADER
I20260926 06:53:06.066277 6370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93675bbb507d4074821057857921a858" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42353 } }
I20260926 06:53:06.066306 6365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:06.066578 6370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93675bbb507d4074821057857921a858. Latest consensus state: current_term: 1 leader_uuid: "93675bbb507d4074821057857921a858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93675bbb507d4074821057857921a858" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42353 } } }
I20260926 06:53:06.066648 6370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:06.066810 6371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93675bbb507d4074821057857921a858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93675bbb507d4074821057857921a858" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42353 } } }
I20260926 06:53:06.066908 6371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:06.067616 6375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:06.068089 6375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:06.069624 6375 catalog_manager.cc:1347] Generated new cluster ID: 443f7bbf9db949369bb2e771ac94e88d
I20260926 06:53:06.069671 6375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:53:06.075785 6368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:06.062010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59038 (local address 127.0.191.254:42353)
0926 06:53:06.062209 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:06.062217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:06.062265 (+ 48us) server_negotiation.cc:407] Connection header received
0926 06:53:06.062816 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:06.062823 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:06.062874 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:06.063110 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:06.063721 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.064585 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.065608 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.066125 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.066550 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:06.069108 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:06.069112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:06.069113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:06.069125 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:06.074164 (+ 5039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.074592 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.074595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.074595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.074677 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.074898 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.074898 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.074899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.075147 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:06.075319 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:06.075401 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 06:53:06.075547 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
I20260926 06:53:06.076529 6375 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:06.077046 6375 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:06.087400 6375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93675bbb507d4074821057857921a858: Generated new TSK 0
I20260926 06:53:06.087584 6375 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:53:06.121753 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42353
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:39511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:06.221727 6391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:06.221923 6391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:06.221947 6391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:06.223102 6391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:06.223143 6391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:06.223201 6391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:53:06.224720 6391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:06.224964 6391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:06.225198 6391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:06.230818 6397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:06.230821 6399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:06.230849 6391 server_base.cc:1034] running on GCE node
W20260926 06:53:06.230795 6396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:06.231215 6391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:06.231470 6391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:06.232645 6391 hybrid_clock.cc:648] HybridClock initialized: now 1790405586232634 us; error 30 us; skew 500 ppm
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:06.235296 6391 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:53:06.235782 6391 webserver.cc:460] Webserver started at http://127.0.191.193:42893/ using document root <none> and password file <none>
I20260926 06:53:06.236028 6391 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:06.236097 6391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:06.236241 6391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:06.237352 6391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "9410e38f49184a5d9cd84bc010c65bcb"
format_stamp: "Formatted at 2026-09-26 06:53:06 on dist-test-slave-ws74"
server_key: "3f762284205580086c363dd79db0cbde"
server_key_iv: "39878ac79bef47575a17b938fd8b6943"
server_key_version: "encryptionkey@0"
I20260926 06:53:06.237783 6391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "9410e38f49184a5d9cd84bc010c65bcb"
format_stamp: "Formatted at 2026-09-26 06:53:06 on dist-test-slave-ws74"
server_key: "3f762284205580086c363dd79db0cbde"
server_key_iv: "39878ac79bef47575a17b938fd8b6943"
server_key_version: "encryptionkey@0"
I20260926 06:53:06.239151 6391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:06.239877 6406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:06.240037 6391 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:06.240124 6391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "9410e38f49184a5d9cd84bc010c65bcb"
format_stamp: "Formatted at 2026-09-26 06:53:06 on dist-test-slave-ws74"
server_key: "3f762284205580086c363dd79db0cbde"
server_key_iv: "39878ac79bef47575a17b938fd8b6943"
server_key_version: "encryptionkey@0"
I20260926 06:53:06.240226 6391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:06.256989 6391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:06.258797 6391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:06.258961 6391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:06.259202 6391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:06.259506 6391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:06.259560 6391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:06.259595 6391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:06.259622 6391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:06.265201 6391 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:33767
I20260926 06:53:06.265239 6519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:33767 every 8 connection(s)
I20260926 06:53:06.265605 6391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:53:06.266947 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 6391
I20260926 06:53:06.267055 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:53:06.267282 767 external_mini_cluster.cc:1428] Setting key 155c08ae0a7faa22461c17fdb79ae1f4
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
I20260926 06:53:06.276080 6368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:06.266300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:48185 (local address 127.0.191.254:42353)
0926 06:53:06.266369 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:06.266371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:06.266618 (+ 247us) server_negotiation.cc:407] Connection header received
0926 06:53:06.267961 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:06.267962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:06.267973 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:06.268035 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:06.268910 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.269485 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.270487 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.270718 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.270795 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:06.273218 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:06.273220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:06.273220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:06.273227 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:06.274674 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.275103 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.275104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.275105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.275171 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.275424 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.275425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.275425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.275544 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:06.275614 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:06.275903 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 06:53:06.275988 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 06:53:06.276257 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:06.266289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:42353 (local address 127.0.191.193:48185)
0926 06:53:06.266471 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:06.266523 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:06.267844 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:06.268120 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:06.268124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:06.268523 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:06.268824 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:06.268829 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.269578 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.269579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:06.270381 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:06.270383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.270800 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.270801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:06.270927 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:06.271466 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:06.271474 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:06.273098 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:06.274774 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:06.274780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:06.274783 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:06.275027 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:06.275244 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:06.275245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:06.275245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:06.275351 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:06.275616 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:06.275623 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:06.275766 (+ 143us) client_negotiation.cc:769] Sending connection context
0926 06:53:06.275871 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 06:53:06.276025 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
I20260926 06:53:06.276762 6520 heartbeater.cc:347] Connected to a master server at 127.0.191.254:42353
I20260926 06:53:06.276866 6520 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:06.277076 6520 heartbeater.cc:511] Master 127.0.191.254:42353 requested a full tablet report, sending...
I20260926 06:53:06.277647 6329 ts_manager.cc:194] Registered new tserver with Master: 9410e38f49184a5d9cd84bc010c65bcb (127.0.191.193:33767)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
I20260926 06:53:06.278275 6329 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:48185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:53:06.292382 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42353
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:39511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:06.371182 6525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:06.371361 6525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:06.371383 6525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:06.372495 6525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:06.372540 6525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:06.372565 6525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:53:06.373942 6525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:06.374109 6525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:06.374274 6525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:53:06.378994 6525 server_base.cc:1034] running on GCE node
W20260926 06:53:06.378979 6533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:06.378971 6530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:06.379132 6531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:06.379431 6525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:06.379657 6525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:06.380825 6525 hybrid_clock.cc:648] HybridClock initialized: now 1790405586380802 us; error 41 us; skew 500 ppm
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:06.383260 6525 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:53:06.383698 6525 webserver.cc:460] Webserver started at http://127.0.191.194:41541/ using document root <none> and password file <none>
I20260926 06:53:06.383922 6525 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:06.383987 6525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:06.384131 6525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:06.385079 6525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "b774b4cb2c534b49b8cf4ca63f35023c"
format_stamp: "Formatted at 2026-09-26 06:53:06 on dist-test-slave-ws74"
server_key: "837de702efb71744d459444ab33e9658"
server_key_iv: "7cc933bf0fc65f3107898e2b5a191197"
server_key_version: "encryptionkey@0"
I20260926 06:53:06.385480 6525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "b774b4cb2c534b49b8cf4ca63f35023c"
format_stamp: "Formatted at 2026-09-26 06:53:06 on dist-test-slave-ws74"
server_key: "837de702efb71744d459444ab33e9658"
server_key_iv: "7cc933bf0fc65f3107898e2b5a191197"
server_key_version: "encryptionkey@0"
I20260926 06:53:06.386696 6525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:53:06.387356 6540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:06.387504 6525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:53:06.387594 6525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "b774b4cb2c534b49b8cf4ca63f35023c"
format_stamp: "Formatted at 2026-09-26 06:53:06 on dist-test-slave-ws74"
server_key: "837de702efb71744d459444ab33e9658"
server_key_iv: "7cc933bf0fc65f3107898e2b5a191197"
server_key_version: "encryptionkey@0"
I20260926 06:53:06.387681 6525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:06.406616 6525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:06.408288 6525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:06.408438 6525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:06.408646 6525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:06.408931 6525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:06.408985 6525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:06.409024 6525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:06.409051 6525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:06.414757 6525 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:33779
I20260926 06:53:06.414806 6653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:33779 every 8 connection(s)
I20260926 06:53:06.415088 6525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:53:06.416734 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 6525
I20260926 06:53:06.416850 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:53:06.417054 767 external_mini_cluster.cc:1428] Setting key a957cd28c59d3d6efe736e609914bc72
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
I20260926 06:53:06.424444 6368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:06.415694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:42745 (local address 127.0.191.254:42353)
0926 06:53:06.415772 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:06.415774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:06.415967 (+ 193us) server_negotiation.cc:407] Connection header received
0926 06:53:06.417002 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:06.417004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:06.417015 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:06.417050 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:06.417912 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.418403 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.419389 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.419601 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.419687 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:06.421740 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:06.421742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:06.421742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:06.421749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:06.423086 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.423477 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.423478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.423479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.423536 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.423795 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.423795 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.423796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.423910 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:06.423984 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:06.424276 (+ 292us) server_negotiation.cc:299] Negotiation successful
0926 06:53:06.424353 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:53:06.424628 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:06.415680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:42353 (local address 127.0.191.194:42745)
0926 06:53:06.415858 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:06.415905 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:06.416859 (+ 954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:06.417241 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:06.417244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:06.417577 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:06.417826 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:06.417831 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.418499 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.418500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:06.419298 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:06.419300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.419695 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.419697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:06.419835 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:06.420314 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:06.420322 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:06.421641 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:06.423177 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:06.423183 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:06.423185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:06.423403 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:06.423609 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:06.423609 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:06.423610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:06.423720 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:06.423987 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:06.423994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:06.424127 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 06:53:06.424251 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 06:53:06.424401 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260926 06:53:06.425173 6654 heartbeater.cc:347] Connected to a master server at 127.0.191.254:42353
I20260926 06:53:06.425279 6654 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:06.425477 6654 heartbeater.cc:511] Master 127.0.191.254:42353 requested a full tablet report, sending...
I20260926 06:53:06.425838 6329 ts_manager.cc:194] Registered new tserver with Master: b774b4cb2c534b49b8cf4ca63f35023c (127.0.191.194:33779)
I20260926 06:53:06.426167 6329 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:42745
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:53:06.440783 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42353
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:39511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:06.519495 6659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:06.519716 6659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:06.519750 6659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:06.520888 6659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:06.520955 6659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:06.520999 6659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:53:06.522468 6659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:06.522713 6659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:06.522936 6659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:06.527614 6665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:06.527628 6664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:06.527583 6667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:06.528007 6659 server_base.cc:1034] running on GCE node
I20260926 06:53:06.528224 6659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:06.528412 6659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:06.529551 6659 hybrid_clock.cc:648] HybridClock initialized: now 1790405586529537 us; error 29 us; skew 500 ppm
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:06.531985 6659 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:53:06.532320 6659 webserver.cc:460] Webserver started at http://127.0.191.195:46297/ using document root <none> and password file <none>
I20260926 06:53:06.532496 6659 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:06.532533 6659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:06.532614 6659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:06.533612 6659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "297bad15d83640aa84a27f725016482a"
format_stamp: "Formatted at 2026-09-26 06:53:06 on dist-test-slave-ws74"
server_key: "90878547f02a572d2fdd40cd896fd330"
server_key_iv: "c09bd9f67a45d152d588f349655349d9"
server_key_version: "encryptionkey@0"
I20260926 06:53:06.533910 6659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "297bad15d83640aa84a27f725016482a"
format_stamp: "Formatted at 2026-09-26 06:53:06 on dist-test-slave-ws74"
server_key: "90878547f02a572d2fdd40cd896fd330"
server_key_iv: "c09bd9f67a45d152d588f349655349d9"
server_key_version: "encryptionkey@0"
I20260926 06:53:06.535079 6659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:06.535708 6674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:06.535867 6659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:06.535938 6659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "297bad15d83640aa84a27f725016482a"
format_stamp: "Formatted at 2026-09-26 06:53:06 on dist-test-slave-ws74"
server_key: "90878547f02a572d2fdd40cd896fd330"
server_key_iv: "c09bd9f67a45d152d588f349655349d9"
server_key_version: "encryptionkey@0"
I20260926 06:53:06.535986 6659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:06.549324 6659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:06.550973 6659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:06.551092 6659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:06.551290 6659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:06.551510 6659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:06.551538 6659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:06.551558 6659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:06.551570 6659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:06.557036 6659 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:45289
I20260926 06:53:06.557113 6787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:45289 every 8 connection(s)
I20260926 06:53:06.557422 6659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:06.565033 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 6659
I20260926 06:53:06.565131 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:53:06.565323 767 external_mini_cluster.cc:1428] Setting key baadaf6dda007d0705f76ae7a345f91a
I20260926 06:53:06.566756 6368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:06.558114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:37181 (local address 127.0.191.254:42353)
0926 06:53:06.558159 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:06.558161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:06.558347 (+ 186us) server_negotiation.cc:407] Connection header received
0926 06:53:06.559282 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:06.559284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:06.559296 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:06.559349 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:06.559945 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.560451 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.561476 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.561680 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.561757 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:06.563690 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:06.563691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:06.563691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:06.563698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:06.565096 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.565770 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.565771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.565772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.565839 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.566116 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.566116 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.566117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.566216 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:06.566307 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:06.566588 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 06:53:06.566683 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:53:06.566977 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:06.558071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:42353 (local address 127.0.191.195:37181)
0926 06:53:06.558213 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:06.558258 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:06.559169 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:06.559414 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:06.559418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:06.559669 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:06.559847 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:06.559851 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.560542 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.560543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:06.561340 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:06.561342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.561762 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.561763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:06.561888 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:06.562281 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:06.562290 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:06.563600 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:06.565302 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:06.565310 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:06.565317 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:06.565683 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:06.565921 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:06.565922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:06.565922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:06.566054 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:06.566318 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:06.566323 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:06.566487 (+ 164us) client_negotiation.cc:769] Sending connection context
0926 06:53:06.566569 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 06:53:06.566718 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260926 06:53:06.567451 6788 heartbeater.cc:347] Connected to a master server at 127.0.191.254:42353
I20260926 06:53:06.567525 6788 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:06.567647 6788 heartbeater.cc:511] Master 127.0.191.254:42353 requested a full tablet report, sending...
I20260926 06:53:06.567997 6329 ts_manager.cc:194] Registered new tserver with Master: 297bad15d83640aa84a27f725016482a (127.0.191.195:45289)
I20260926 06:53:06.568364 6329 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:37181
I20260926 06:53:06.577217 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/test-admin.keytab.
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:06.593156 767 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 06:53:06.594763 6795 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:06.594011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46646 (local address 127.0.191.193:33767)
0926 06:53:06.594158 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:06.594163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:06.594236 (+ 73us) server_negotiation.cc:407] Connection header received
0926 06:53:06.594279 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:06.594281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:06.594295 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:06.594360 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:06.594685 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46646: BlockingRecv error: recv got EOF from 127.0.0.1:46646 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
W20260926 06:53:06.594877 767 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/krb5cc)
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:06.595845 767 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:06.596719 767 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.193@KRBTEST.COM
I20260926 06:53:06.604310 6795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:06.597237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46648 (local address 127.0.191.193:33767)
0926 06:53:06.597299 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:06.597301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:06.597347 (+ 46us) server_negotiation.cc:407] Connection header received
0926 06:53:06.597424 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:06.597425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:06.597438 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:06.597484 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:06.597862 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.598563 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.599251 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.599542 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.599596 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:06.601693 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:06.601699 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:06.601699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:06.601705 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:06.603148 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.603551 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.603553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.603553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.603602 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.603815 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.603815 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.603816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.603926 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:06.604026 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:06.604154 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 06:53:06.604206 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260926 06:53:06.605999 6795 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:06.605446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46660 (local address 127.0.191.193:33767)
0926 06:53:06.605482 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:06.605484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:06.605547 (+ 63us) server_negotiation.cc:407] Connection header received
0926 06:53:06.605604 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:06.605605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:06.605617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:06.605684 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:06.605946 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46660: BlockingRecv error: recv got EOF from 127.0.0.1:46660 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 06:53:06.606010 767 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/krb5cc' of type FILE
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:06.606904 767 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:06.607749 767 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.193@KRBTEST.COM
I20260926 06:53:06.614688 6795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:06.608250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46676 (local address 127.0.191.193:33767)
0926 06:53:06.608302 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:06.608304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:06.608373 (+ 69us) server_negotiation.cc:407] Connection header received
0926 06:53:06.608409 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:06.608409 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:06.608418 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:06.608467 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:06.608750 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.609504 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.610126 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.610317 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.610370 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:06.612223 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:06.612224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:06.612224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:06.612231 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:06.613595 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.613932 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.613933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.613933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.613987 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.614193 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.614194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.614195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.614315 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:06.614406 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:06.614520 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 06:53:06.614590 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260926 06:53:06.616101 6795 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:06.615602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46686 (local address 127.0.191.193:33767)
0926 06:53:06.615639 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:06.615641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:06.615674 (+ 33us) server_negotiation.cc:407] Connection header received
0926 06:53:06.615736 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:06.615737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:06.615744 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:06.615785 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:06.616034 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46686: BlockingRecv error: recv got EOF from 127.0.0.1:46686 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260926 06:53:06.616158 767 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790405559478510-767-0/krb5kdc/krb5cc' of type FILE
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:06.617050 767 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:06.617897 767 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.193@KRBTEST.COM
I20260926 06:53:06.623718 6795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:06.618320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46702 (local address 127.0.191.193:33767)
0926 06:53:06.618376 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:06.618378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:06.618387 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:06.618441 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:06.618441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:06.618448 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:06.618490 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:06.618777 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.619199 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.619791 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:06.619955 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:06.620000 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:06.621721 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:06.621722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:06.621722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:06.621735 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:06.622827 (+ 1092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.623116 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.623116 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.623117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.623160 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:06.623313 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:06.623313 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:06.623313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:06.623387 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:06.623449 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:06.623544 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 06:53:06.623592 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 06:53:06.624353 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 6391
I20260926 06:53:06.628922 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 6525
I20260926 06:53:06.634512 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 6659
I20260926 06:53:06.638405 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 6297
2026-09-26T06:53:06Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1877 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6808](info): set up 2 sockets
Sep 26 06:53:06 dist-test-slave-ws74 krb5kdc[6808](info): commencing operation
krb5kdc: starting...
W20260926 06:53:07.741521 767 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:53:07 dist-test-slave-ws74 krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:53:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.254.keytab.
Entry for principal oryx/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.254.keytab.
I20260926 06:53:07.856254 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:37691
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.254.keytab
--principal=oryx/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:37691
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:07.934471 6821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:07.934687 6821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:07.934723 6821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:07.935843 6821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:07.935904 6821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:07.935930 6821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:07.935957 6821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:07.935986 6821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:07.937361 6821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:37691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.254.keytab
--principal=oryx/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:37691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:07.937652 6821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:07.937851 6821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:07.942355 6826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:07.942343 6827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:07.942343 6829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:07.942432 6821 server_base.cc:1034] running on GCE node
I20260926 06:53:07.942896 6821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:07.943182 6821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:07.944344 6821 hybrid_clock.cc:648] HybridClock initialized: now 1790405587944331 us; error 27 us; skew 500 ppm
Sep 26 06:53:07 dist-test-slave-ws74 krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405587, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:07.946672 6821 init.cc:377] Logged in from keytab as oryx/127.0.191.254@KRBTEST.COM (short username oryx)
I20260926 06:53:07.947052 6821 webserver.cc:460] Webserver started at http://127.0.191.254:39795/ using document root <none> and password file <none>
I20260926 06:53:07.947254 6821 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:07.947316 6821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:07.947458 6821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:07.948387 6821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "957ffa2a68c546fe923af9f52bc212bb"
format_stamp: "Formatted at 2026-09-26 06:53:07 on dist-test-slave-ws74"
server_key: "ec5d9e378c9e78c84244ac240791a9f7"
server_key_iv: "5afaaf1c8c15d56717b586093bf2364f"
server_key_version: "encryptionkey@0"
I20260926 06:53:07.948716 6821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "957ffa2a68c546fe923af9f52bc212bb"
format_stamp: "Formatted at 2026-09-26 06:53:07 on dist-test-slave-ws74"
server_key: "ec5d9e378c9e78c84244ac240791a9f7"
server_key_iv: "5afaaf1c8c15d56717b586093bf2364f"
server_key_version: "encryptionkey@0"
I20260926 06:53:07.950027 6821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:53:07.950682 6836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:07.950843 6821 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:07.950938 6821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "957ffa2a68c546fe923af9f52bc212bb"
format_stamp: "Formatted at 2026-09-26 06:53:07 on dist-test-slave-ws74"
server_key: "ec5d9e378c9e78c84244ac240791a9f7"
server_key_iv: "5afaaf1c8c15d56717b586093bf2364f"
server_key_version: "encryptionkey@0"
I20260926 06:53:07.951005 6821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:07.977546 6821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:07.979451 6821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:07.979622 6821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:07.983639 6821 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:37691
I20260926 06:53:07.983673 6888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:37691 every 8 connection(s)
I20260926 06:53:07.984027 6821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:07.985073 6889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:07.987272 6889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb: Bootstrap starting.
I20260926 06:53:07.987774 6889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:07.987998 6889 log.cc:824] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:07.988639 6889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb: No bootstrap required, opened a new log
I20260926 06:53:07.990192 6889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } }
I20260926 06:53:07.990324 6889 raft_consensus.cc:374] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:07.990346 6889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 957ffa2a68c546fe923af9f52bc212bb, State: Initialized, Role: FOLLOWER
I20260926 06:53:07.990474 6889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } }
I20260926 06:53:07.990564 6889 raft_consensus.cc:388] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:07.990586 6889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:07.990648 6889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:07.990738 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 6821
I20260926 06:53:07.990871 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:07.991053 767 external_mini_cluster.cc:1428] Setting key c677b41da6b452e2686e860e2dbb83dd
I20260926 06:53:07.991628 6889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } }
I20260926 06:53:07.991770 6889 leader_election.cc:302] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 957ffa2a68c546fe923af9f52bc212bb; no voters:
I20260926 06:53:07.991922 6889 leader_election.cc:288] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:07.991940 6894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:07.992038 6894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [term 1 LEADER]: Becoming Leader. State: Replica: 957ffa2a68c546fe923af9f52bc212bb, State: Running, Role: LEADER
I20260926 06:53:07.992141 6894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } }
I20260926 06:53:07.992177 6889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:07.992462 6894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } }
I20260926 06:53:07.992519 6894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:07.992636 6894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 957ffa2a68c546fe923af9f52bc212bb. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } }
I20260926 06:53:07.992686 6894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:07.992801 6899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:07.993584 6899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:07.996042 6899 catalog_manager.cc:1347] Generated new cluster ID: 9145caa6548345cb96badc48689bd591
I20260926 06:53:07.996094 6899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:53:07 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
I20260926 06:53:08.002571 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:07.991511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58046 (local address 127.0.191.254:37691)
0926 06:53:07.991682 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:07.991694 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:07.991747 (+ 53us) server_negotiation.cc:407] Connection header received
0926 06:53:07.991871 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:07.991878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:07.991932 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:07.992124 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:07.993027 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:07.994888 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:07.995668 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:07.996257 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:07.996353 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:07.999026 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:07.999030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:07.999031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:07.999042 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.000946 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.001336 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.001340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.001341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.001507 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.001709 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.001710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.001710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.002007 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.002175 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.002240 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.002372 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260926 06:53:08.007611 6899 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:08.008312 6899 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:08.015336 6899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb: Generated new TSK 0
I20260926 06:53:08.015513 6899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab.
Entry for principal oryx/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab.
I20260926 06:53:08.044095 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:37691
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab
--principal=oryx/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:08.122556 6915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:08.122757 6915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:08.122780 6915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:08.123925 6915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:08.123965 6915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:08.124015 6915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:53:08.125411 6915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab
--principal=oryx/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:37691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:08.125646 6915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:08.125939 6915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:08.130765 6921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:08.130774 6920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:08.130774 6923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:08.130788 6915 server_base.cc:1034] running on GCE node
I20260926 06:53:08.131135 6915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:08.131372 6915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:08.132535 6915 hybrid_clock.cc:648] HybridClock initialized: now 1790405588132526 us; error 46 us; skew 500 ppm
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:08.135007 6915 init.cc:377] Logged in from keytab as oryx/127.0.191.193@KRBTEST.COM (short username oryx)
I20260926 06:53:08.135414 6915 webserver.cc:460] Webserver started at http://127.0.191.193:34179/ using document root <none> and password file <none>
I20260926 06:53:08.135625 6915 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:08.135696 6915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:08.135844 6915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:08.136824 6915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "6a8e80e1b50742878a46ee35b9e33c36"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "20a41de65cef98541987c2ffb9a8a7a5"
server_key_iv: "532a10c355aa4c2a27d857846d2451bd"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.137197 6915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "6a8e80e1b50742878a46ee35b9e33c36"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "20a41de65cef98541987c2ffb9a8a7a5"
server_key_iv: "532a10c355aa4c2a27d857846d2451bd"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.138481 6915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:53:08.139218 6930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.139387 6915 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:53:08.139482 6915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "6a8e80e1b50742878a46ee35b9e33c36"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "20a41de65cef98541987c2ffb9a8a7a5"
server_key_iv: "532a10c355aa4c2a27d857846d2451bd"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.139559 6915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:08.147459 6915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:08.149106 6915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:08.149259 6915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:08.149528 6915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:08.149816 6915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:08.149869 6915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.149906 6915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:08.149936 6915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.155568 6915 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:41465
I20260926 06:53:08.155602 7043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:41465 every 8 connection(s)
I20260926 06:53:08.155889 6915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:53:08.158290 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 6915
I20260926 06:53:08.158389 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:53:08.158592 767 external_mini_cluster.cc:1428] Setting key 0a8e37cc76c5b27e33ade8d593828d8f
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
I20260926 06:53:08.166373 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.156583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.193:59533)
0926 06:53:08.156718 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.156758 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.157235 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.157508 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.157511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.158101 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.158409 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.158415 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.159273 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.159274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.160579 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.160582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.161078 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.161079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.161173 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.161702 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.161711 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.163266 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.165105 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.165110 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.165112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.165318 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.165544 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.165545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.165545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.165639 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.165928 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.165931 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.166033 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.166110 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.166229 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260926 06:53:08.166375 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.156585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:59533 (local address 127.0.191.254:37691)
0926 06:53:08.156647 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.156649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.156839 (+ 190us) server_negotiation.cc:407] Connection header received
0926 06:53:08.157349 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.157350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.157362 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.157438 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.158519 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.159172 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.160683 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.160999 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.161070 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.163357 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.163359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.163359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.163368 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.165017 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.165406 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.165408 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.165408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.165470 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.165701 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.165702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.165703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.165860 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.165935 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.166161 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.166253 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:53:08.167044 7044 heartbeater.cc:347] Connected to a master server at 127.0.191.254:37691
I20260926 06:53:08.167146 7044 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:08.167328 7044 heartbeater.cc:511] Master 127.0.191.254:37691 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.0.191.194@KRBTEST.COM; defaulting to no policy
I20260926 06:53:08.167824 6853 ts_manager.cc:194] Registered new tserver with Master: 6a8e80e1b50742878a46ee35b9e33c36 (127.0.191.193:41465)
I20260926 06:53:08.168416 6853 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:59533
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab.
Entry for principal oryx/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab.
I20260926 06:53:08.183974 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:37691
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab
--principal=oryx/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:08.266597 7049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:08.266790 7049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:08.266814 7049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:08.268020 7049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:08.268059 7049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:08.268085 7049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:53:08.269562 7049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab
--principal=oryx/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:37691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:08.269783 7049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:08.270004 7049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:08.274780 7057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:08.274787 7054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:08.274844 7055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:08.274822 7049 server_base.cc:1034] running on GCE node
I20260926 06:53:08.275178 7049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:08.275363 7049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:08.276526 7049 hybrid_clock.cc:648] HybridClock initialized: now 1790405588276518 us; error 29 us; skew 500 ppm
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:08.278882 7049 init.cc:377] Logged in from keytab as oryx/127.0.191.194@KRBTEST.COM (short username oryx)
I20260926 06:53:08.279312 7049 webserver.cc:460] Webserver started at http://127.0.191.194:44961/ using document root <none> and password file <none>
I20260926 06:53:08.279526 7049 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:08.279568 7049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:08.279737 7049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:08.280700 7049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "888b433ce9094d4295a686cebda88449"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "b8f36de343b2f3a8a5a936d903c73b7e"
server_key_iv: "a9f8a7912f97210ea8bfef2dc69c2005"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.281038 7049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "888b433ce9094d4295a686cebda88449"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "b8f36de343b2f3a8a5a936d903c73b7e"
server_key_iv: "a9f8a7912f97210ea8bfef2dc69c2005"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.282367 7049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:53:08.283105 7064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.283274 7049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.283342 7049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "888b433ce9094d4295a686cebda88449"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "b8f36de343b2f3a8a5a936d903c73b7e"
server_key_iv: "a9f8a7912f97210ea8bfef2dc69c2005"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.283429 7049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:08.298972 7049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:08.300554 7049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:08.300696 7049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:08.300892 7049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:08.301159 7049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:08.301194 7049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.301214 7049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:08.301265 7049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.306684 7049 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:37313
I20260926 06:53:08.306734 7177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:37313 every 8 connection(s)
I20260926 06:53:08.307006 7049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:53:08.308122 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 7049
I20260926 06:53:08.308203 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:53:08.308408 767 external_mini_cluster.cc:1428] Setting key 92d947c96998d9828f831cf329ed1154
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.0.191.195@KRBTEST.COM; defaulting to no policy
I20260926 06:53:08.316985 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.307650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:41591 (local address 127.0.191.254:37691)
0926 06:53:08.307735 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.307737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.307886 (+ 149us) server_negotiation.cc:407] Connection header received
0926 06:53:08.308370 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.308371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.308385 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.308455 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.309553 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.310333 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.311815 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.312014 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.312100 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.314189 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.314191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.314191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.314199 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.315597 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.315985 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.315986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.315987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.316050 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.316299 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.316300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.316300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.316451 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.316541 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.316805 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.316895 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 06:53:08.317137 7180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.307610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.194:41591)
0926 06:53:08.307775 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.307807 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.308278 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.308552 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.308556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.309284 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.309474 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.309478 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.310425 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.310426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.311724 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.311726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.312119 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.312119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.312245 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.312737 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.312745 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.314093 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.315686 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.315693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.315695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.315908 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.316117 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.316118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.316118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.316225 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.316542 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.316549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.316682 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.316785 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.316925 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
I20260926 06:53:08.317644 7178 heartbeater.cc:347] Connected to a master server at 127.0.191.254:37691
I20260926 06:53:08.317723 7178 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:08.317920 7178 heartbeater.cc:511] Master 127.0.191.254:37691 requested a full tablet report, sending...
I20260926 06:53:08.318303 6853 ts_manager.cc:194] Registered new tserver with Master: 888b433ce9094d4295a686cebda88449 (127.0.191.194:37313)
I20260926 06:53:08.318687 6853 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:41591
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab.
Entry for principal oryx/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab.
I20260926 06:53:08.331938 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:37691
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab
--principal=oryx/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:08.410467 7183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:08.410655 7183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:08.410678 7183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:08.411791 7183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:08.411829 7183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:08.411867 7183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:53:08.413236 7183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab
--principal=oryx/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:37691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:08.413491 7183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:08.413709 7183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:08.418414 7191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:08.418432 7188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:08.418409 7189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:08.418731 7183 server_base.cc:1034] running on GCE node
I20260926 06:53:08.418928 7183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:08.419144 7183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:08.420300 7183 hybrid_clock.cc:648] HybridClock initialized: now 1790405588420291 us; error 39 us; skew 500 ppm
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:08.422745 7183 init.cc:377] Logged in from keytab as oryx/127.0.191.195@KRBTEST.COM (short username oryx)
I20260926 06:53:08.423221 7183 webserver.cc:460] Webserver started at http://127.0.191.195:38385/ using document root <none> and password file <none>
I20260926 06:53:08.423441 7183 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:08.423507 7183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:08.423661 7183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:08.424639 7183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "2c998e0fe06946d1ac61842d0d139930"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "5e55ee54a347445208c1ba6db476bfbb"
server_key_iv: "9284eeec500d21c1cec0e7fc5409f26d"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.424983 7183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "2c998e0fe06946d1ac61842d0d139930"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "5e55ee54a347445208c1ba6db476bfbb"
server_key_iv: "9284eeec500d21c1cec0e7fc5409f26d"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.426316 7183 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:53:08.426980 7198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.427138 7183 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:08.427240 7183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "2c998e0fe06946d1ac61842d0d139930"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "5e55ee54a347445208c1ba6db476bfbb"
server_key_iv: "9284eeec500d21c1cec0e7fc5409f26d"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.427318 7183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:08.446606 7183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:08.448872 7183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:08.449033 7183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:08.449259 7183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:08.449604 7183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:08.449656 7183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.449697 7183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:08.449723 7183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.455550 7183 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:41139
I20260926 06:53:08.455597 7311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:41139 every 8 connection(s)
I20260926 06:53:08.455919 7183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:53:08.456087 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 7183
I20260926 06:53:08.456192 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:53:08.456400 767 external_mini_cluster.cc:1428] Setting key 747fc47e896d6e7822eb90479e5c9591
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
I20260926 06:53:08.464439 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.456651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:36333 (local address 127.0.191.254:37691)
0926 06:53:08.456710 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.456712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.456906 (+ 194us) server_negotiation.cc:407] Connection header received
0926 06:53:08.457563 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.457563 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.457572 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.457618 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.458502 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.459014 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.459893 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.460063 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.460109 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.462154 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.462156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.462156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.462162 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.463306 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.463638 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.463639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.463639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.463694 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.463954 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.463955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.463955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.464047 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.464094 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.464324 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.464378 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:53:08.464692 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.456598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.195:36333)
0926 06:53:08.456770 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.456814 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.457464 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.457667 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.457670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.458224 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.458409 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.458412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.459084 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.459085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.459822 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.459824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.460126 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.460126 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.460218 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.460710 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.460721 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.462078 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.463373 (+ 1295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.463377 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.463379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.463576 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.463778 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.463779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.463779 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.463890 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.464104 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.464109 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.464214 (+ 105us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.464296 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.464415 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260926 06:53:08.465202 7312 heartbeater.cc:347] Connected to a master server at 127.0.191.254:37691
I20260926 06:53:08.465277 7312 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:08.465508 7312 heartbeater.cc:511] Master 127.0.191.254:37691 requested a full tablet report, sending...
I20260926 06:53:08.465884 6853 ts_manager.cc:194] Registered new tserver with Master: 2c998e0fe06946d1ac61842d0d139930 (127.0.191.195:41139)
I20260926 06:53:08.466215 6853 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:36333
I20260926 06:53:08.468146 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.191.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.191.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.191.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.253.keytab.
Entry for principal oryx/127.0.191.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.253.keytab.
I20260926 06:53:08.491954 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.253:43773
--webserver_interface=127.0.191.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.253.keytab
--principal=oryx/127.0.191.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:37691,127.0.191.253:43773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:08.570330 7317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:08.570523 7317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:08.570544 7317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:08.571681 7317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:08.571723 7317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:08.571738 7317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:08.571750 7317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:08.571799 7317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:08.573202 7317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:37691,127.0.191.253:43773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.253.keytab
--principal=oryx/127.0.191.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.253:43773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.191.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
W20260926 06:53:08.573323 7317 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.0.191.254:37691,127.0.191.253:43773'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 06:53:08.573485 7317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:08.573709 7317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:08.578245 7322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:08.578272 7323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:08.578253 7325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:08.578456 7317 server_base.cc:1034] running on GCE node
I20260926 06:53:08.578698 7317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:08.578925 7317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:08.580058 7317 hybrid_clock.cc:648] HybridClock initialized: now 1790405588580040 us; error 33 us; skew 500 ppm
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:08.582472 7317 init.cc:377] Logged in from keytab as oryx/127.0.191.253@KRBTEST.COM (short username oryx)
I20260926 06:53:08.582891 7317 webserver.cc:460] Webserver started at http://127.0.191.253:33091/ using document root <none> and password file <none>
I20260926 06:53:08.583117 7317 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:08.583185 7317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:08.583335 7317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:08.584374 7317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/data/instance:
uuid: "362c18ce7b844446b05453009f59c391"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "bd6b686afebb57695915256ea4507c9f"
server_key_iv: "540a25cd0d070ca4a2ff9c43660bb7c2"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.584724 7317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/wal/instance:
uuid: "362c18ce7b844446b05453009f59c391"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "bd6b686afebb57695915256ea4507c9f"
server_key_iv: "540a25cd0d070ca4a2ff9c43660bb7c2"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.586227 7317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:53:08.586907 7332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.587078 7317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:08.587144 7317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/wal
uuid: "362c18ce7b844446b05453009f59c391"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "bd6b686afebb57695915256ea4507c9f"
server_key_iv: "540a25cd0d070ca4a2ff9c43660bb7c2"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.587234 7317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:08.614722 7317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:08.617043 7317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:08.617200 7317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:08.620870 7317 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.253:43773
I20260926 06:53:08.620923 7384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.253:43773 every 8 connection(s)
I20260926 06:53:08.621245 7317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/data/info.pb
I20260926 06:53:08.622203 7385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:08.623700 7385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:53:08.626327 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 7317
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.253@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.191.253@KRBTEST.COM
I20260926 06:53:08.633451 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.624372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58050 (local address 127.0.191.254:37691)
0926 06:53:08.624453 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.624455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.624634 (+ 179us) server_negotiation.cc:407] Connection header received
0926 06:53:08.625071 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.625072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.625081 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.625141 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.625826 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.626335 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.627771 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.627969 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.628059 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.630544 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.630545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.630546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.630553 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.631970 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.632383 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.632384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.632385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.632444 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.632727 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.632728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.632729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.632863 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.632920 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.633251 (+ 331us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.633332 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 06:53:08.633632 7387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.624361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:58050)
0926 06:53:08.624508 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.624547 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.624992 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.625215 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.625218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.625537 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.625752 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.625756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.626466 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.626466 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.627703 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.627706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.628080 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.628081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.628203 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.628678 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.628688 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.630458 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.632070 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.632076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.632078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.632307 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.632511 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.632512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.632513 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.632654 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.632918 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.632924 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.633081 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.633206 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.633339 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260926 06:53:08.634232 7385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:53:08.635551 7389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.626800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49358 (local address 127.0.191.253:43773)
0926 06:53:08.626942 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.626946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.626990 (+ 44us) server_negotiation.cc:407] Connection header received
0926 06:53:08.627007 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.627009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.627023 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.627074 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.627720 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.628423 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.629495 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.629982 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.630057 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.632420 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.632426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.632427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.632433 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.634319 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.634754 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.634757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.634757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.634821 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.635014 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.635015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.635015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.635168 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.635313 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.635381 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.635453 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":94,"threads_started":1}
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.253@KRBTEST.COM for oryx/127.0.191.253@KRBTEST.COM
I20260926 06:53:08.640956 7387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.634504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:43773 (local address 127.0.0.1:49362)
0926 06:53:08.634673 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.634682 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.634725 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.634882 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.634883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.635020 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.635066 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.635068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.635906 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.635907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.636490 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.636492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.636811 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.636811 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.636865 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.637149 (+ 284us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.637152 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.638225 (+ 1073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.640249 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.640250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.640251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.640396 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.640557 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.640558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.640558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.640604 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.640763 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.640764 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.640791 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.640836 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.640901 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260926 06:53:08.641069 7390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.634564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49362 (local address 127.0.191.253:43773)
0926 06:53:08.634681 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.634682 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.634720 (+ 38us) server_negotiation.cc:407] Connection header received
0926 06:53:08.634778 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.634779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.634789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.634854 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.635138 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.635830 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.636554 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.636748 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.636805 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.638299 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.638300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.638300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.638305 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.640181 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.640456 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.640457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.640457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.640504 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.640640 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.640641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.640641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.640722 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.640765 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.640849 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.640943 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":99,"thread_start_us":79,"threads_started":1}
I20260926 06:53:08.642302 7385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: Bootstrap starting.
I20260926 06:53:08.642885 7385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:08.643121 7385 log.cc:824] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:08.643729 7385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: No bootstrap required, opened a new log
I20260926 06:53:08.645002 7385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } }
I20260926 06:53:08.645113 7385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:08.645135 7385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 362c18ce7b844446b05453009f59c391, State: Initialized, Role: FOLLOWER
I20260926 06:53:08.645256 7385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } }
I20260926 06:53:08.645509 7392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } } }
I20260926 06:53:08.645570 7385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:08.645591 7392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 06:53:08.647135 7403 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:53:08.647189 7403 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:53:08.648588 7317 master_runner.cc:428] Detected that this master 362c18ce7b844446b05453009f59c391 is joining an existing cluster
I20260926 06:53:08.648623 7317 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.191.253:43773
I20260926 06:53:08.656450 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.649620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:43773 (local address 127.0.0.1:49378)
0926 06:53:08.649838 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.649847 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.649893 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.650018 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.650019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.650245 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.650325 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.650327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.651216 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.651217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.652004 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.652005 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.652438 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.652438 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.652494 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.653052 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.653056 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.653462 (+ 406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.655244 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.655245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.655246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.655410 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.655617 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.655618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.655618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.655670 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.656096 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.656097 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.656135 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.656201 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.656300 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":91,"threads_started":1}
I20260926 06:53:08.656642 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.649685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58056 (local address 127.0.191.254:37691)
0926 06:53:08.649788 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.649790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.650006 (+ 216us) server_negotiation.cc:407] Connection header received
0926 06:53:08.650080 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.650081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.650090 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.650145 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.650451 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.650932 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.651968 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.652270 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.652343 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.653805 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.653806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.653807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.653814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.655407 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.655823 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.655824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.655824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.655873 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.656086 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.656087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.656087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.656196 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.656271 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.656485 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.656561 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 06:53:08.656527 7390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.649745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49378 (local address 127.0.191.253:43773)
0926 06:53:08.649780 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.649782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.649905 (+ 123us) server_negotiation.cc:407] Connection header received
0926 06:53:08.649927 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.649928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.649937 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.650000 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.650406 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.651138 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.652097 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.652336 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.652396 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.653544 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.653545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.653546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.653553 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.655155 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.655492 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.655493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.655493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.655555 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.655754 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.655755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.655755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.655877 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.655942 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.656238 (+ 296us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.656376 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":18}
I20260926 06:53:08.656980 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.649628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:58056)
0926 06:53:08.649947 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.649954 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.650026 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.650146 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.650147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.650317 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.650362 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.650364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.651014 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.651015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.651880 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.651882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.652359 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.652360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.652420 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.652901 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.652905 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.653462 (+ 557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.655499 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.655500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.655500 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.655749 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.655944 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.655945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.655945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.656019 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.656295 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.656296 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.656332 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.656790 (+ 458us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.656878 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":72,"threads_started":1}
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405587, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.254@KRBTEST.COM for oryx/127.0.191.253@KRBTEST.COM
I20260926 06:53:08.665586 7390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.658996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49392 (local address 127.0.191.253:43773)
0926 06:53:08.659033 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.659035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.659155 (+ 120us) server_negotiation.cc:407] Connection header received
0926 06:53:08.659249 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.659249 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.659258 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.659301 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.659691 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.660117 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.660853 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.661041 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.661123 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.663192 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.663193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.663194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.663200 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.664427 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.664808 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.664809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.664809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.664864 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.665069 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.665070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.665070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.665168 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.665242 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.665436 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.665504 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:53:08.665618 7411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.658941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:43773 (local address 127.0.0.1:49392)
0926 06:53:08.659050 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.659088 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.659178 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.659315 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.659319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.659572 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.659613 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.659615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.660196 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.660198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.660771 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.660773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.661127 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.661128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.661211 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.661657 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.661662 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.663107 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.664516 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.664522 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.664524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.664714 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.664942 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.664943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.664943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.665006 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.665242 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.665249 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.665296 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.665411 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.665519 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":58,"threads_started":1}
I20260926 06:53:08.666271 6853 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.191.253:43773: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "362c18ce7b844446b05453009f59c391"
member_type: NON_VOTER
last_known_addr {
host: "127.0.191.253"
port: 43773
}
attrs {
promote: true
}
}
dest_uuid: "957ffa2a68c546fe923af9f52bc212bb"
cas_config_opid_index: -1
I20260926 06:53:08.666566 6853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } }
I20260926 06:53:08.667013 7413 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 362c18ce7b844446b05453009f59c391 (127.0.191.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } }
I20260926 06:53:08.667306 7359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:08.667904 7413 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.191.253:43773
I20260926 06:53:08.668083 7317 master.cc:537] Master@127.0.191.253:43773 shutting down...
I20260926 06:53:08.668144 7412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } }
I20260926 06:53:08.668186 7359 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 FOLLOWER]: Refusing update from remote peer 957ffa2a68c546fe923af9f52bc212bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 06:53:08.668221 7412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:08.668488 7412 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 06:53:08.669147 6838 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260926 06:53:08.669914 6838 consensus_peers.cc:487] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb -> Peer 362c18ce7b844446b05453009f59c391 (127.0.191.253:43773): Couldn't send request to peer 362c18ce7b844446b05453009f59c391. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 06:53:08.671155 7317 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:08.671254 7317 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:08.671308 7317 tablet_replica.cc:330] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: stopping tablet replica
I20260926 06:53:08.683913 7317 master.cc:552] Master@127.0.191.253:43773 shutdown complete.
I20260926 06:53:08.683964 7317 master_runner.cc:305] Clearing existing system tablet
I20260926 06:53:08.684092 7317 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:08.684140 7317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:08.684815 7317 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260926 06:53:08.685348 7417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.685505 7317 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:08.685565 7317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.685590 7317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/wal
uuid: "362c18ce7b844446b05453009f59c391"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "bd6b686afebb57695915256ea4507c9f"
server_key_iv: "540a25cd0d070ca4a2ff9c43660bb7c2"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.686331 7317 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:08.687980 7317 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 06:53:08.688046 7317 log.cc:1196] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 06:53:08.688230 7317 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: Deleting consensus metadata
I20260926 06:53:08.688342 7317 master_runner.cc:315] Copying system tablet from 127.0.191.254:37691
I20260926 06:53:08.688920 7317 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: tablet copy: Beginning tablet copy session from remote peer at address 127.0.191.254:37691
I20260926 06:53:08.695111 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.689257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58068 (local address 127.0.191.254:37691)
0926 06:53:08.689303 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.689311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.689366 (+ 55us) server_negotiation.cc:407] Connection header received
0926 06:53:08.689466 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.689467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.689478 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.689534 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.689862 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.690354 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.691082 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.691284 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.691367 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.692399 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.692401 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.692401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.692408 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.694016 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.694361 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.694362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.694362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.694422 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.694623 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.694624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.694624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.694733 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.694807 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.694950 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.695021 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 06:53:08.695139 7423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.689181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:58068)
0926 06:53:08.689300 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.689316 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.689402 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.689539 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.689540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.689730 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.689779 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.689781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.690439 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.690440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.690996 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.690998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.691367 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.691369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.691434 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.691924 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.691928 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.692299 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.694103 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.694104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.694105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.694279 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.694484 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.694485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.694485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.694543 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.694798 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.694799 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.694842 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.694926 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.695023 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":71,"threads_started":1}
I20260926 06:53:08.695616 6873 tablet_copy_service.cc:140] P 957ffa2a68c546fe923af9f52bc212bb: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 362c18ce7b844446b05453009f59c391 ({username='oryx', principal='oryx/127.0.191.253@KRBTEST.COM'} at 127.0.0.1:58068)
I20260926 06:53:08.695703 6873 tablet_copy_service.cc:161] P 957ffa2a68c546fe923af9f52bc212bb: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 362c18ce7b844446b05453009f59c391 at {username='oryx', principal='oryx/127.0.191.253@KRBTEST.COM'} at 127.0.0.1:58068: session id = 362c18ce7b844446b05453009f59c391-00000000000000000000000000000000
I20260926 06:53:08.696296 6873 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 06:53:08.696858 7317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 06:53:08.697786 7317 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: tablet copy: Starting download of 0 data blocks...
I20260926 06:53:08.697921 7317 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: tablet copy: Starting download of 1 WAL segments...
I20260926 06:53:08.698762 7317 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 06:53:08.699510 6873 tablet_copy_service.cc:342] P 957ffa2a68c546fe923af9f52bc212bb: Request end of tablet copy session 362c18ce7b844446b05453009f59c391-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.191.253@KRBTEST.COM'} at 127.0.0.1:58068
I20260926 06:53:08.699566 6873 tablet_copy_service.cc:433] P 957ffa2a68c546fe923af9f52bc212bb: ending tablet copy session 362c18ce7b844446b05453009f59c391-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 362c18ce7b844446b05453009f59c391
I20260926 06:53:08.702127 7317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:08.703224 7424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:08.703253 7425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:08.703301 7427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:08.703665 7317 server_base.cc:1034] running on GCE node
I20260926 06:53:08.703765 7317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:08.703909 7317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:08.705045 7317 hybrid_clock.cc:648] HybridClock initialized: now 1790405588705035 us; error 24 us; skew 500 ppm
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:08.706465 7317 init.cc:377] Logged in from keytab as oryx/127.0.191.253@KRBTEST.COM (short username oryx)
I20260926 06:53:08.706712 7317 webserver.cc:460] Webserver started at http://127.0.191.253:41299/ using document root <none> and password file <none>
I20260926 06:53:08.706838 7317 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:08.706892 7317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:08.707472 7317 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:08.707966 7434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.708105 7317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.708155 7317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/wal
uuid: "362c18ce7b844446b05453009f59c391"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "bd6b686afebb57695915256ea4507c9f"
server_key_iv: "540a25cd0d070ca4a2ff9c43660bb7c2"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.708206 7317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:08.749346 7317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:08.750928 7317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:08.753921 7317 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.253:43773
I20260926 06:53:08.753952 7486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.253:43773 every 8 connection(s)
I20260926 06:53:08.754248 7317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-1/data/info.pb
I20260926 06:53:08.754982 7487 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:53:08.755378 7487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: Bootstrap starting.
I20260926 06:53:08.757659 7487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:53:08.757870 7487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: Bootstrap complete.
I20260926 06:53:08.758049 7487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } }
I20260926 06:53:08.758136 7487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 362c18ce7b844446b05453009f59c391, State: Initialized, Role: LEARNER
I20260926 06:53:08.758183 7487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } }
I20260926 06:53:08.758333 7487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:08.758328 7490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } }
I20260926 06:53:08.758436 7490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: This master's current role is: LEARNER
I20260926 06:53:08.759727 7501 catalog_manager.cc:1259] Loaded cluster ID: 9145caa6548345cb96badc48689bd591
I20260926 06:53:08.759790 7501 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: loading cluster ID for follower catalog manager: success
I20260926 06:53:08.760825 7501 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: acquiring CA information for follower catalog manager: success
I20260926 06:53:08.761098 7501 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:53:08.793725 7502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.787478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49408 (local address 127.0.191.253:43773)
0926 06:53:08.787617 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.787618 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.787625 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:53:08.787646 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.787647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.787655 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.787723 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.788295 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.788927 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.789742 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.789926 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.790010 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.791175 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.791176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.791177 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.791184 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.792591 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.792927 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.792928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.792929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.792999 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.793188 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.793189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.793189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.793311 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.793433 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.793540 (+ 107us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.793620 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for oryx/127.0.191.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.191.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.191.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.252.keytab.
Entry for principal oryx/127.0.191.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.252.keytab.
I20260926 06:53:08.816440 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.252:34135
--webserver_interface=127.0.191.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.252.keytab
--principal=oryx/127.0.191.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:37691,127.0.191.253:43773,127.0.191.252:34135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:08.897176 7505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:08.897367 7505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:08.897447 7505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:08.898586 7505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:08.898625 7505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:08.898639 7505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:08.898695 7505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:08.898711 7505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:08.900197 7505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:37691,127.0.191.253:43773,127.0.191.252:34135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.252.keytab
--principal=oryx/127.0.191.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.252:34135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.191.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:08.900370 7505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:08.900604 7505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:08.905690 7511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:08.905719 7513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:08.905704 7510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:08.905746 7505 server_base.cc:1034] running on GCE node
I20260926 06:53:08.906162 7505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:08.906481 7505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:08.907630 7505 hybrid_clock.cc:648] HybridClock initialized: now 1790405588907624 us; error 33 us; skew 500 ppm
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:08.910063 7505 init.cc:377] Logged in from keytab as oryx/127.0.191.252@KRBTEST.COM (short username oryx)
I20260926 06:53:08.910512 7505 webserver.cc:460] Webserver started at http://127.0.191.252:37117/ using document root <none> and password file <none>
I20260926 06:53:08.910739 7505 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:08.910805 7505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:08.910934 7505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:08.911916 7505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/data/instance:
uuid: "f40b9326c65f4616a70fbf25924e152e"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "916b6e538783cd5c399d44e692b87df2"
server_key_iv: "9e9f066e235e33689e1eff2b5c927b5f"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.912264 7505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/wal/instance:
uuid: "f40b9326c65f4616a70fbf25924e152e"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "916b6e538783cd5c399d44e692b87df2"
server_key_iv: "9e9f066e235e33689e1eff2b5c927b5f"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.913553 7505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:53:08.914211 7520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.914386 7505 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:08.914474 7505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/wal
uuid: "f40b9326c65f4616a70fbf25924e152e"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "916b6e538783cd5c399d44e692b87df2"
server_key_iv: "9e9f066e235e33689e1eff2b5c927b5f"
server_key_version: "encryptionkey@0"
I20260926 06:53:08.914573 7505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:08.925081 7505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:08.926823 7505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:08.926976 7505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:08.931324 7505 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.252:34135
I20260926 06:53:08.931351 7572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.252:34135 every 8 connection(s)
I20260926 06:53:08.931687 7505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/data/info.pb
I20260926 06:53:08.932621 7573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:08.934198 7573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.252@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
I20260926 06:53:08.941008 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 7505
I20260926 06:53:08.942871 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.934965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58078 (local address 127.0.191.254:37691)
0926 06:53:08.935035 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.935037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.935052 (+ 15us) server_negotiation.cc:407] Connection header received
0926 06:53:08.935544 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.935545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.935558 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.935608 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.936184 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.936740 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.937796 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.937990 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.938074 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.940010 (+ 1936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.940011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.940012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.940019 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.941380 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.941861 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.941863 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.941863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.941923 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.942164 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.942165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.942165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.942281 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.942359 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.942694 (+ 335us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.942784 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 06:53:08.943032 7575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.934776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:58078)
0926 06:53:08.934933 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.934974 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.935433 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.935678 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.935681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.935915 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.936090 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.936094 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.936828 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.936829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.937694 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.937696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.938100 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.938102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.938211 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.938644 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.938651 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.939918 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.941478 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.941484 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.941486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.941773 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.941987 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.941988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.941989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.942097 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.942349 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.942354 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.942545 (+ 191us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.942660 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.942804 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":59,"threads_started":1}
I20260926 06:53:08.943507 7573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.191.252@KRBTEST.COM
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.252@KRBTEST.COM for oryx/127.0.191.253@KRBTEST.COM
I20260926 06:53:08.950274 7575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.943762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:43773 (local address 127.0.0.1:49420)
0926 06:53:08.943823 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.943832 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.943868 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.943997 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.943997 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.944120 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.944161 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.944162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.944809 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.944810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.945285 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.945286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.945751 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.945752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.945819 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.946210 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.946215 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.947546 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.949159 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.949160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.949160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.949356 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.949574 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.949575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.949575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.949647 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.949922 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.949923 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.949964 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.950037 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.950136 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260926 06:53:08.950353 7502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.943838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49420 (local address 127.0.191.253:43773)
0926 06:53:08.943896 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.943898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.943906 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:53:08.943929 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.943930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.943942 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.943986 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.944242 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.944704 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.945365 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.945679 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.945752 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.947656 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.947657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.947658 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.947665 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.949089 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.949441 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.949442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.949443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.949503 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.949716 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.949716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.949717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.949839 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.949913 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.950074 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.950155 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 06:53:08.950356 7576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.941590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40410 (local address 127.0.191.252:34135)
0926 06:53:08.941745 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.941749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.941773 (+ 24us) server_negotiation.cc:407] Connection header received
0926 06:53:08.941795 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.941797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.941809 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.941895 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.942450 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.943397 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.944213 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.944517 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.944604 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.947226 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.947232 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.947232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.947238 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.949007 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.949458 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.949461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.949461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.949510 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.949716 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.949716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.949717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.949873 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.950001 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.950074 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.950190 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260926 06:53:08.950650 7573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:53:08 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405588, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.252@KRBTEST.COM for oryx/127.0.191.252@KRBTEST.COM
I20260926 06:53:08.955932 7575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.950873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:34135 (local address 127.0.0.1:40424)
0926 06:53:08.950939 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.950948 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.950995 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.951093 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.951094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.951188 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.951210 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.951211 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.951717 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.951718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.952139 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.952140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.952439 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.952440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.952472 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.952839 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.952843 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.953999 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.955253 (+ 1254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.955254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.955254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.955401 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.955543 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.955543 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.955543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.955581 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.955727 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.955733 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.955760 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.955803 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.955862 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 06:53:08.956023 7576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.950905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40424 (local address 127.0.191.252:34135)
0926 06:53:08.950938 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.950940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.950983 (+ 43us) server_negotiation.cc:407] Connection header received
0926 06:53:08.951047 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.951048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.951054 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.951091 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.951255 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.951654 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.952202 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.952383 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.952430 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.954061 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.954062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.954062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.954067 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.955189 (+ 1122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.955462 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.955462 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.955462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.955505 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.955611 (+ 106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.955612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.955612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.955694 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.955736 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.955819 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.955901 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":16}
I20260926 06:53:08.957123 7573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: Bootstrap starting.
I20260926 06:53:08.957912 7573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:08.958201 7573 log.cc:824] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:08.958909 7573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: No bootstrap required, opened a new log
I20260926 06:53:08.960774 7573 raft_consensus.cc:348] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } }
I20260926 06:53:08.960888 7573 raft_consensus.cc:374] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:08.960909 7573 raft_consensus.cc:729] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f40b9326c65f4616a70fbf25924e152e, State: Initialized, Role: FOLLOWER
I20260926 06:53:08.961021 7573 consensus_queue.cc:260] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } }
I20260926 06:53:08.961315 7573 sys_catalog.cc:564] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:08.961274 7578 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } } }
I20260926 06:53:08.961409 7578 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: This master's current role is: FOLLOWER
W20260926 06:53:08.962821 7589 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:53:08.962882 7589 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:53:08.965093 7505 master_runner.cc:428] Detected that this master f40b9326c65f4616a70fbf25924e152e is joining an existing cluster
I20260926 06:53:08.965140 7505 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.191.252:34135
I20260926 06:53:08.972657 7502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.966334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49436 (local address 127.0.191.253:43773)
0926 06:53:08.966378 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.966379 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.966577 (+ 198us) server_negotiation.cc:407] Connection header received
0926 06:53:08.966670 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.966672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.966680 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.966726 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.967064 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.967525 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.968580 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.968753 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.968796 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.969765 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.969766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.969767 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.969772 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.971455 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.971807 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.971808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.971809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.971858 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.972029 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.972030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.972030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.972139 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.972194 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.972475 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.972557 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:53:08.973112 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.966123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:43773 (local address 127.0.0.1:49436)
0926 06:53:08.966510 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.966521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.966609 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.966731 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.966732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.966898 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.966982 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.966984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.967598 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.967599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.968482 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.968484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.968827 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.968828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.968898 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.969279 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.969284 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.969690 (+ 406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.971530 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.971531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.971532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.971730 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.971922 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.971922 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.971923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.971972 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.972240 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.972241 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.972326 (+ 85us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.972875 (+ 549us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.972984 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":37,"thread_start_us":105,"threads_started":1}
I20260926 06:53:08.998765 7595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.966191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:34135 (local address 127.0.0.1:40440)
0926 06:53:08.970290 (+ 4099us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.970301 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.970365 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.970508 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.970509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.970695 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.970745 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.970747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.971607 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.971607 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.972402 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.972404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.972861 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.972862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.972915 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.973328 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.973333 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.973702 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.975323 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.975323 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.975324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.997774 (+ 22450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.998007 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.998008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.998009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.998092 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:08.998388 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:08.998389 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:08.998451 (+ 62us) client_negotiation.cc:769] Sending connection context
0926 06:53:08.998517 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 06:53:08.998636 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4076,"thread_start_us":82,"threads_started":1}
I20260926 06:53:08.998840 7576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.966134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40440 (local address 127.0.191.252:34135)
0926 06:53:08.966530 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.966532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.970377 (+ 3845us) server_negotiation.cc:407] Connection header received
0926 06:53:08.970408 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.970409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.970418 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.970474 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.970823 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.971521 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.972489 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.972760 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.972797 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.973801 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.973802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.973803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.973809 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.975237 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.997873 (+ 22636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.997875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.997875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.997941 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:08.998167 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:08.998168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:08.998168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:08.998316 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:08.998377 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:08.998596 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 06:53:08.998707 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":379}
I20260926 06:53:09.000697 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.966377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:58088)
0926 06:53:08.970452 (+ 4075us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:08.970460 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:08.970804 (+ 344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:08.971017 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:08.971018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:08.971175 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:08.971251 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.971252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.972179 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.972180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.974222 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:08.974224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.974990 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.974991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:08.975064 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.975454 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:08.975457 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:08.998004 (+ 22547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:08.999835 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:08.999836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:08.999836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:08.999986 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.000190 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.000190 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.000191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.000236 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.000484 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:09.000485 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:09.000515 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.000556 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.000626 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4054,"thread_start_us":56,"threads_started":1}
I20260926 06:53:09.000751 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:08.966530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58088 (local address 127.0.191.254:37691)
0926 06:53:08.966573 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:08.966576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:08.970505 (+ 3929us) server_negotiation.cc:407] Connection header received
0926 06:53:08.970896 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:08.970897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:08.970910 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:08.970941 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:08.971342 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.972093 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.974359 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:08.974903 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:08.974975 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:08.998103 (+ 23128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:08.998105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:08.998105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:08.998113 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:08.999753 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.000064 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.000065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.000066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.000126 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.000300 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.000300 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.000301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.000428 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.000492 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.000587 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.000660 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:53:09.001564 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 6915
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405587, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.254@KRBTEST.COM for oryx/127.0.191.252@KRBTEST.COM
I20260926 06:53:09.007274 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:41465
--local_ip_for_outbound_sockets=127.0.191.193
--tserver_master_addrs=127.0.191.254:37691
--webserver_port=34179
--webserver_interface=127.0.191.193
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab
--principal=oryx/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.191.254:37691,127.0.191.253:43773
--tserver_master_addrs=127.0.191.254:37691,127.0.191.253:43773,127.0.191.252:34135 with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:53:09.009949 7576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.003189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40448 (local address 127.0.191.252:34135)
0926 06:53:09.003222 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.003223 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.003257 (+ 34us) server_negotiation.cc:407] Connection header received
0926 06:53:09.003332 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.003333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.003340 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.003407 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.003674 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.004119 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.004803 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.005005 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.005086 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.007179 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.007180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.007180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.007186 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.008771 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.009181 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.009181 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.009182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.009239 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.009464 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.009464 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.009465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.009575 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.009634 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.009794 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.009865 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:53:09.010016 7411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.003154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:34135 (local address 127.0.0.1:40448)
0926 06:53:09.003203 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.003215 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.003273 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.003413 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.003413 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.003557 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:09.003593 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.003595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.004212 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.004213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.004725 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.004727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.005086 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.005087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.005152 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.005903 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:09.005907 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:09.007090 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:09.008901 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.008902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.008903 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.009104 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.009312 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.009312 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.009313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.009366 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.009630 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:09.009631 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:09.009679 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.009757 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.009865 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 06:53:09.010464 6853 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.191.252:34135: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f40b9326c65f4616a70fbf25924e152e"
member_type: NON_VOTER
last_known_addr {
host: "127.0.191.252"
port: 34135
}
attrs {
promote: true
}
}
dest_uuid: "957ffa2a68c546fe923af9f52bc212bb"
cas_config_opid_index: 5
I20260926 06:53:09.010689 6853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } }
I20260926 06:53:09.011132 7412 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f40b9326c65f4616a70fbf25924e152e (127.0.191.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } } }
I20260926 06:53:09.012072 7412 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.191.252:34135
I20260926 06:53:09.012135 7412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } } }
I20260926 06:53:09.012200 7412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:09.012773 7547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:09.013682 7547 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 FOLLOWER]: Refusing update from remote peer 957ffa2a68c546fe923af9f52bc212bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 06:53:09.013917 7413 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:09.014333 7505 master.cc:537] Master@127.0.191.252:34135 shutting down...
I20260926 06:53:09.017120 7411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.012388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:43773 (local address 127.0.0.1:49442)
0926 06:53:09.012458 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.012467 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.012496 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.012715 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.012716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.012718 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:09.012756 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.012757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.013833 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.013834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.015157 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.015160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.016831 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.016832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.016909 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.016933 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.016990 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.017034 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 06:53:09.017786 7502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.012468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49442 (local address 127.0.191.253:43773)
0926 06:53:09.012512 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.012514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.012522 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:53:09.012549 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.012550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.012575 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.012647 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:09.012836 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.013666 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.015912 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.016601 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.016694 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.016779 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.017678 (+ 899us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.017716 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:53:09.018013 7461 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 LEARNER]: Refusing update from remote peer 957ffa2a68c546fe923af9f52bc212bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 06:53:09.018190 7547 raft_consensus.cc:858] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } }
I20260926 06:53:09.018226 7599 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260926 06:53:09.018273 7547 raft_consensus.cc:1565] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } }
I20260926 06:53:09.018698 7547 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 362c18ce7b844446b05453009f59c391 (127.0.191.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } }
I20260926 06:53:09.018759 7461 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f40b9326c65f4616a70fbf25924e152e (127.0.191.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } } }
I20260926 06:53:09.019021 7600 mvcc.cc:205] Tried to move back new op lower bound from 7333501288509550592 to 7333501288416243712. Current Snapshot: MvccSnapshot[applied={T|T < 7333501288426467328}]
I20260926 06:53:09.019762 7490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 957ffa2a68c546fe923af9f52bc212bb. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } } }
I20260926 06:53:09.019837 7490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: This master's current role is: LEARNER
I20260926 06:53:09.019866 7490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } } }
I20260926 06:53:09.019850 7578 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 957ffa2a68c546fe923af9f52bc212bb. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } }
I20260926 06:53:09.019903 7490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: This master's current role is: LEARNER
I20260926 06:53:09.019917 7578 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260926 06:53:09.019953 7578 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } }
I20260926 06:53:09.019995 7578 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260926 06:53:09.020022 7578 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: NON_VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: true } } }
I20260926 06:53:09.020054 7578 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260926 06:53:09.020483 7599 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb: attempting to promote NON_VOTER 362c18ce7b844446b05453009f59c391 to VOTER
I20260926 06:53:09.020656 7599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } }
I20260926 06:53:09.021346 7461 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 LEARNER]: Refusing update from remote peer 957ffa2a68c546fe923af9f52bc212bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 06:53:09.021566 7412 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260926 06:53:09.021644 6840 consensus_peers.cc:487] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb -> Peer f40b9326c65f4616a70fbf25924e152e (127.0.191.252:34135): Couldn't send request to peer f40b9326c65f4616a70fbf25924e152e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 06:53:09.022207 7413 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 362c18ce7b844446b05453009f59c391 (127.0.191.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } } }
I20260926 06:53:09.022341 7505 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260926 06:53:09.022435 7505 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260926 06:53:09.022475 7505 tablet_replica.cc:330] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: stopping tablet replica
I20260926 06:53:09.022454 7461 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 362c18ce7b844446b05453009f59c391 (127.0.191.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } } }
I20260926 06:53:09.022987 7413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } } }
I20260926 06:53:09.023067 7413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:09.023115 7413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } } }
I20260926 06:53:09.023154 7413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:09.023504 7608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } } }
I20260926 06:53:09.023645 7608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:53:09.035648 7505 master.cc:552] Master@127.0.191.252:34135 shutdown complete.
I20260926 06:53:09.035698 7505 master_runner.cc:305] Clearing existing system tablet
I20260926 06:53:09.035825 7505 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:09.035868 7505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:09.036574 7505 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:09.037128 7611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.037302 7505 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:09.037362 7505 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.037405 7505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/wal
uuid: "f40b9326c65f4616a70fbf25924e152e"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "916b6e538783cd5c399d44e692b87df2"
server_key_iv: "9e9f066e235e33689e1eff2b5c927b5f"
server_key_version: "encryptionkey@0"
I20260926 06:53:09.038033 7505 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:09.039420 7505 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 06:53:09.039485 7505 log.cc:1196] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 06:53:09.039709 7505 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: Deleting consensus metadata
I20260926 06:53:09.039829 7505 master_runner.cc:315] Copying system tablet from 127.0.191.254:37691
I20260926 06:53:09.040396 7505 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.191.254:37691
I20260926 06:53:09.048168 7617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.040674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:58102)
0926 06:53:09.041507 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.041519 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.041587 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.041736 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.041737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.041931 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:09.041982 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.041984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.042924 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.042925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.043763 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.043766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.044245 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.044246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.044316 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.044837 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:09.044842 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:09.045219 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:09.047094 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.047095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.047096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.047318 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.047527 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.047527 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.047528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.047595 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.047862 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:09.047863 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:09.047910 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.047983 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.048078 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":62,"threads_started":1}
I20260926 06:53:09.048190 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.040745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58102 (local address 127.0.191.254:37691)
0926 06:53:09.040794 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.040796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.041575 (+ 779us) server_negotiation.cc:407] Connection header received
0926 06:53:09.041653 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.041654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.041666 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.041731 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.042065 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.042840 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.043854 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.044145 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.044223 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.045323 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.045325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.045325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.045332 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.047006 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.047395 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.047396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.047397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.047455 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.047664 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.047665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.047666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.047790 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.047858 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.048019 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.048096 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:53:09.048790 6873 tablet_copy_service.cc:140] P 957ffa2a68c546fe923af9f52bc212bb: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f40b9326c65f4616a70fbf25924e152e ({username='oryx', principal='oryx/127.0.191.252@KRBTEST.COM'} at 127.0.0.1:58102)
I20260926 06:53:09.048846 6873 tablet_copy_service.cc:161] P 957ffa2a68c546fe923af9f52bc212bb: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f40b9326c65f4616a70fbf25924e152e at {username='oryx', principal='oryx/127.0.191.252@KRBTEST.COM'} at 127.0.0.1:58102: session id = f40b9326c65f4616a70fbf25924e152e-00000000000000000000000000000000
I20260926 06:53:09.049438 6873 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 06:53:09.049902 7505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 06:53:09.050976 7505 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: tablet copy: Starting download of 0 data blocks...
I20260926 06:53:09.051095 7505 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: tablet copy: Starting download of 1 WAL segments...
I20260926 06:53:09.051831 7505 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 06:53:09.052649 6873 tablet_copy_service.cc:342] P 957ffa2a68c546fe923af9f52bc212bb: Request end of tablet copy session f40b9326c65f4616a70fbf25924e152e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.191.252@KRBTEST.COM'} at 127.0.0.1:58102
I20260926 06:53:09.052706 6873 tablet_copy_service.cc:433] P 957ffa2a68c546fe923af9f52bc212bb: ending tablet copy session f40b9326c65f4616a70fbf25924e152e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f40b9326c65f4616a70fbf25924e152e
I20260926 06:53:09.055419 7505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:09.056593 7621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:09.056718 7618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:09.057000 7619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:09.057279 7505 server_base.cc:1034] running on GCE node
I20260926 06:53:09.057395 7505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:09.057519 7505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:09.058630 7505 hybrid_clock.cc:648] HybridClock initialized: now 1790405589058621 us; error 25 us; skew 500 ppm
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405589, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:09.060174 7505 init.cc:377] Logged in from keytab as oryx/127.0.191.252@KRBTEST.COM (short username oryx)
I20260926 06:53:09.060421 7505 webserver.cc:460] Webserver started at http://127.0.191.252:33331/ using document root <none> and password file <none>
I20260926 06:53:09.060525 7505 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:09.060550 7505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:09.061211 7505 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:09.061784 7628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.061898 7505 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.061940 7505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/wal
uuid: "f40b9326c65f4616a70fbf25924e152e"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "916b6e538783cd5c399d44e692b87df2"
server_key_iv: "9e9f066e235e33689e1eff2b5c927b5f"
server_key_version: "encryptionkey@0"
I20260926 06:53:09.061995 7505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:09.078348 7505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:09.080113 7505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:09.082979 7505 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.252:34135
I20260926 06:53:09.083355 7505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/master-2/data/info.pb
I20260926 06:53:09.089630 7680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.252:34135 every 8 connection(s)
I20260926 06:53:09.090356 7681 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:53:09.090896 7681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: Bootstrap starting.
I20260926 06:53:09.093730 7681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:53:09.093967 7681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: Bootstrap complete.
I20260926 06:53:09.094193 7681 raft_consensus.cc:348] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } }
I20260926 06:53:09.094297 7681 raft_consensus.cc:729] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f40b9326c65f4616a70fbf25924e152e, State: Initialized, Role: LEARNER
I20260926 06:53:09.094340 7681 consensus_queue.cc:260] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } }
I20260926 06:53:09.094583 7681 sys_catalog.cc:564] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:09.094561 7684 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } } }
I20260926 06:53:09.094787 7684 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: This master's current role is: LEARNER
I20260926 06:53:09.096172 7695 catalog_manager.cc:1259] Loaded cluster ID: 9145caa6548345cb96badc48689bd591
I20260926 06:53:09.096280 7695 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: loading cluster ID for follower catalog manager: success
I20260926 06:53:09.097604 7695 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: acquiring CA information for follower catalog manager: success
I20260926 06:53:09.097892 7695 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 06:53:09.111827 7598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:09.112051 7598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:09.112088 7598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:09.113240 7598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:09.113305 7598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:09.113350 7598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:53:09.114884 7598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab
--principal=oryx/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:41465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=34179
--tserver_master_addrs=127.0.191.254:37691,127.0.191.253:43773,127.0.191.252:34135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:09.115144 7598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:09.115370 7598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:09.120961 7703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:09.120988 7701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:09.121019 7598 server_base.cc:1034] running on GCE node
W20260926 06:53:09.120992 7700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:09.121421 7598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:09.121709 7598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:09.122877 7598 hybrid_clock.cc:648] HybridClock initialized: now 1790405589122864 us; error 29 us; skew 500 ppm
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405589, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:09.125229 7598 init.cc:377] Logged in from keytab as oryx/127.0.191.193@KRBTEST.COM (short username oryx)
I20260926 06:53:09.125680 7598 webserver.cc:460] Webserver started at http://127.0.191.193:34179/ using document root <none> and password file <none>
I20260926 06:53:09.125860 7598 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:09.125909 7598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:09.127138 7598 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:53:09.127620 7710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.127808 7598 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.127875 7598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "6a8e80e1b50742878a46ee35b9e33c36"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "20a41de65cef98541987c2ffb9a8a7a5"
server_key_iv: "532a10c355aa4c2a27d857846d2451bd"
server_key_version: "encryptionkey@0"
I20260926 06:53:09.128214 7598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:09.148619 7598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:09.150812 7598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:09.150961 7598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:09.151182 7598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:09.151469 7598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:09.151500 7598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.151520 7598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:09.151562 7598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.158814 7598 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:41465
I20260926 06:53:09.158923 7823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:41465 every 8 connection(s)
I20260926 06:53:09.159305 7598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:53:09.162899 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 7598
I20260926 06:53:09.163074 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 7049
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405589, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405589, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.252@KRBTEST.COM
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405589, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.253@KRBTEST.COM
I20260926 06:53:09.168598 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:37313
--local_ip_for_outbound_sockets=127.0.191.194
--tserver_master_addrs=127.0.191.254:37691
--webserver_port=44961
--webserver_interface=127.0.191.194
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab
--principal=oryx/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.191.254:37691,127.0.191.253:43773
--tserver_master_addrs=127.0.191.254:37691,127.0.191.253:43773,127.0.191.252:34135 with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:53:09.170928 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.160370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:36917 (local address 127.0.191.254:37691)
0926 06:53:09.160423 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.160425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.160652 (+ 227us) server_negotiation.cc:407] Connection header received
0926 06:53:09.160925 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.160926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.160937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.160968 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.161763 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.162269 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.164482 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.164756 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.164823 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.167631 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.167632 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.167633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.167641 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.169328 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.169826 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.169827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.169827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.169890 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.170180 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.170181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.170181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.170330 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.170387 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.170745 (+ 358us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.170826 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 06:53:09.171128 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.160281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.193:36917)
0926 06:53:09.160577 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.160585 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.160831 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.161080 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.161080 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.161424 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:09.161664 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.161670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.162369 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.162370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.164394 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.164396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.164848 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.164849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.164924 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.165620 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:09.165629 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:09.167518 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:09.169434 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.169441 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.169444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.169729 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.169969 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.169970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.169970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.170098 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.170423 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:09.170429 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:09.170587 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.170691 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.170863 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":51,"thread_start_us":59,"threads_started":1}
I20260926 06:53:09.171938 7825 heartbeater.cc:347] Connected to a master server at 127.0.191.254:37691
I20260926 06:53:09.172034 7825 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:09.172214 7825 heartbeater.cc:511] Master 127.0.191.254:37691 requested a full tablet report, sending...
I20260926 06:53:09.172380 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.159979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:34135 (local address 127.0.191.193:43937)
0926 06:53:09.160161 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.160205 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.160806 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.161062 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.161066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.161411 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:09.161663 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.161675 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.163398 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.163399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.164390 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.164393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.164785 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.164786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.164924 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.166618 (+ 1694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:09.166623 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:09.168605 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:09.171066 (+ 2461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.171068 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.171068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.171451 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.171692 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.171692 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.171693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.171755 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.172041 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:09.172041 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:09.172081 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.172153 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.172258 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":4,"thread_start_us":111,"threads_started":1}
I20260926 06:53:09.172605 6853 ts_manager.cc:194] Re-registered known tserver with Master: 6a8e80e1b50742878a46ee35b9e33c36 (127.0.191.193:41465)
I20260926 06:53:09.172647 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.159980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:43773 (local address 127.0.191.193:50661)
0926 06:53:09.160467 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.160477 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.160824 (+ 347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.161495 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.161496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.161639 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:09.161687 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.161688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.162369 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.162370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.165957 (+ 3587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.165959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.166477 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.166478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.166544 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.166945 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:09.166950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:09.169365 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:09.171621 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.171622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.171622 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.171848 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.172041 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.172041 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.172042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.172093 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.172386 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:09.172387 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:09.172424 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.172484 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.172574 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":35,"thread_start_us":82,"threads_started":1}
I20260926 06:53:09.172633 7830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.160216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:43937 (local address 127.0.191.252:34135)
0926 06:53:09.160436 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.160438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.160450 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:53:09.160911 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.160913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.160928 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.160996 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.162370 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.163313 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.164512 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.164702 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.164783 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.168699 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.168701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.168701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.168708 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.170637 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.171541 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.171542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.171542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.171600 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.171818 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.171819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.171819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.171950 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.172013 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.172185 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.172324 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":73,"threads_started":1}
I20260926 06:53:09.172966 7502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.159956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:50661 (local address 127.0.191.253:43773)
0926 06:53:09.160016 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.160018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.160530 (+ 512us) server_negotiation.cc:407] Connection header received
0926 06:53:09.160920 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.160922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.160934 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.160967 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.161810 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.162262 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.166048 (+ 3786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.166356 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.166424 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.169452 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.169454 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.169454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.169462 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.171276 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.171917 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.171918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.171918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.171971 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.172168 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.172169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.172169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.172301 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.172353 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.172808 (+ 455us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.172879 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 06:53:09.173121 6853 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:36917
I20260926 06:53:09.173134 7824 heartbeater.cc:347] Connected to a master server at 127.0.191.252:34135
I20260926 06:53:09.173185 7824 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:09.173257 7826 heartbeater.cc:347] Connected to a master server at 127.0.191.253:43773
I20260926 06:53:09.173266 7824 heartbeater.cc:511] Master 127.0.191.252:34135 requested a full tablet report, sending...
I20260926 06:53:09.173307 7826 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:09.173357 7826 heartbeater.cc:511] Master 127.0.191.253:43773 requested a full tablet report, sending...
I20260926 06:53:09.173633 7645 ts_manager.cc:194] Registered new tserver with Master: 6a8e80e1b50742878a46ee35b9e33c36 (127.0.191.193:41465)
I20260926 06:53:09.173686 7451 ts_manager.cc:194] Registered new tserver with Master: 6a8e80e1b50742878a46ee35b9e33c36 (127.0.191.193:41465)
W20260926 06:53:09.254114 7832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:09.254331 7832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:09.254369 7832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:09.255550 7832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:09.255618 7832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:09.255676 7832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:53:09.257226 7832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab
--principal=oryx/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:37313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=44961
--tserver_master_addrs=127.0.191.254:37691,127.0.191.253:43773,127.0.191.252:34135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:09.257633 7832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:09.257870 7832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:09.262765 7837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:09.262818 7838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:09.262840 7840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:09.263089 7832 server_base.cc:1034] running on GCE node
I20260926 06:53:09.263314 7832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:09.263553 7832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:09.264688 7832 hybrid_clock.cc:648] HybridClock initialized: now 1790405589264668 us; error 36 us; skew 500 ppm
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405589, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:09.267019 7832 init.cc:377] Logged in from keytab as oryx/127.0.191.194@KRBTEST.COM (short username oryx)
I20260926 06:53:09.267446 7832 webserver.cc:460] Webserver started at http://127.0.191.194:44961/ using document root <none> and password file <none>
I20260926 06:53:09.267669 7832 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:09.267746 7832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:09.268970 7832 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:09.269568 7847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.269726 7832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.269812 7832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "888b433ce9094d4295a686cebda88449"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "b8f36de343b2f3a8a5a936d903c73b7e"
server_key_iv: "a9f8a7912f97210ea8bfef2dc69c2005"
server_key_version: "encryptionkey@0"
I20260926 06:53:09.270113 7832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:09.282173 7832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:09.283838 7832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:09.283983 7832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:09.284180 7832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:09.284459 7832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:09.284510 7832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.284547 7832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:09.284574 7832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.290316 7832 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:37313
I20260926 06:53:09.290361 7960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:37313 every 8 connection(s)
I20260926 06:53:09.290663 7832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:53:09.296415 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 7832
I20260926 06:53:09.296583 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 7183
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405589, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405589, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.253@KRBTEST.COM
I20260926 06:53:09.303382 7502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.291323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:53351 (local address 127.0.191.253:43773)
0926 06:53:09.291419 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.291421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.291787 (+ 366us) server_negotiation.cc:407] Connection header received
0926 06:53:09.292813 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.292814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.292827 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.292878 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.295045 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.295773 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.296743 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.296997 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.297048 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.299782 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.299783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.299783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.299789 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.301068 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.301496 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.301497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.301498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.301557 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.302623 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.302624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.302625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.302790 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.302852 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.303199 (+ 347us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.303286 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 06:53:09.303610 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.291301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:43773 (local address 127.0.191.194:53351)
0926 06:53:09.291715 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.291726 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.292656 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.292945 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.292950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.293472 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:09.293662 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.293674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.295861 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.295862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.296673 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.296675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.297122 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.297123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.297178 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.297598 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:09.297602 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:09.299700 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:09.301150 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.301157 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.301159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.301425 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.301622 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.301623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.301623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.301752 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.302880 (+ 1128us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:09.302886 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:09.303061 (+ 175us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.303162 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.303333 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":8,"thread_start_us":86,"threads_started":1}
I20260926 06:53:09.303648 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.291374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:39815 (local address 127.0.191.254:37691)
0926 06:53:09.291456 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.291458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.291680 (+ 222us) server_negotiation.cc:407] Connection header received
0926 06:53:09.292833 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.292835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.292845 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.292878 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.294362 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.294912 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.295966 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.296288 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.296330 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.300679 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.300682 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.300682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.300690 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.302519 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.302833 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.302834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.302835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.302895 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.303093 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.303094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.303095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.303221 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.303291 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.303470 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.303561 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 06:53:09.304069 7966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.291300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.194:39815)
0926 06:53:09.291518 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.291558 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.292709 (+ 1151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.292945 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.292949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.293476 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:09.293661 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.293666 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.295006 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.295007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.295883 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.295884 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.296418 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.296419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.296481 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.297150 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:09.297159 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:09.300312 (+ 3153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:09.302588 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.302589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.302589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.302755 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.302959 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.302960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.302960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.303029 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.303318 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:09.303319 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:09.303358 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.303863 (+ 505us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.303970 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":79,"threads_started":1}
I20260926 06:53:09.304301 7963 heartbeater.cc:347] Connected to a master server at 127.0.191.253:43773
I20260926 06:53:09.304311 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:41139
--local_ip_for_outbound_sockets=127.0.191.195
--tserver_master_addrs=127.0.191.254:37691
--webserver_port=38385
--webserver_interface=127.0.191.195
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab
--principal=oryx/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.191.254:37691,127.0.191.253:43773
--tserver_master_addrs=127.0.191.254:37691,127.0.191.253:43773,127.0.191.252:34135 with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:53:09.304411 7963 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:09.304600 7963 heartbeater.cc:511] Master 127.0.191.253:43773 requested a full tablet report, sending...
I20260926 06:53:09.305099 7451 ts_manager.cc:194] Registered new tserver with Master: 888b433ce9094d4295a686cebda88449 (127.0.191.194:37313)
I20260926 06:53:09.305743 7962 heartbeater.cc:347] Connected to a master server at 127.0.191.254:37691
I20260926 06:53:09.305804 7962 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:09.305873 7962 heartbeater.cc:511] Master 127.0.191.254:37691 requested a full tablet report, sending...
I20260926 06:53:09.306094 6853 ts_manager.cc:194] Re-registered known tserver with Master: 888b433ce9094d4295a686cebda88449 (127.0.191.194:37313)
I20260926 06:53:09.306644 6853 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:39815
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405589, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.252@KRBTEST.COM
I20260926 06:53:09.310513 7830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.291739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:60967 (local address 127.0.191.252:34135)
0926 06:53:09.291803 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.291804 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.291928 (+ 124us) server_negotiation.cc:407] Connection header received
0926 06:53:09.292831 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.292832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.292846 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.292877 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.293788 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.294284 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.295476 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.295726 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.295806 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.307617 (+ 11811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.307618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.307619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.307628 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.309359 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.309810 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.309811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.309811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.309865 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.310067 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.310068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.310068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.310165 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.310213 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.310378 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.310452 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 06:53:09.310644 7965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.291582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:34135 (local address 127.0.191.194:60967)
0926 06:53:09.291884 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.291892 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.292656 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.292950 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.292951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.293472 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:09.293662 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.293668 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.294404 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.294405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.295385 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.295387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.295814 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.295815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.295945 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.305548 (+ 9603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:09.305554 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:09.307538 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:09.309483 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.309484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.309485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.309736 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.309933 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.309934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.309935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.310003 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.310226 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:09.310227 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:09.310275 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.310355 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.310481 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":33,"thread_start_us":79,"threads_started":1}
I20260926 06:53:09.311002 7961 heartbeater.cc:347] Connected to a master server at 127.0.191.252:34135
I20260926 06:53:09.311072 7961 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:09.311151 7961 heartbeater.cc:511] Master 127.0.191.252:34135 requested a full tablet report, sending...
I20260926 06:53:09.311452 7645 ts_manager.cc:194] Registered new tserver with Master: 888b433ce9094d4295a686cebda88449 (127.0.191.194:37313)
W20260926 06:53:09.385833 7968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:09.386065 7968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:09.386112 7968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:09.387329 7968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:09.387387 7968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:09.387429 7968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:53:09.389014 7968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab
--principal=oryx/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:41139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=38385
--tserver_master_addrs=127.0.191.254:37691,127.0.191.253:43773,127.0.191.252:34135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:09.389271 7968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:09.389555 7968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:09.394517 7973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:09.394518 7976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:09.394670 7968 server_base.cc:1034] running on GCE node
W20260926 06:53:09.394697 7974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:09.394925 7968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:09.395185 7968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:09.396373 7968 hybrid_clock.cc:648] HybridClock initialized: now 1790405589396352 us; error 36 us; skew 500 ppm
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405589, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:09.398802 7968 init.cc:377] Logged in from keytab as oryx/127.0.191.195@KRBTEST.COM (short username oryx)
I20260926 06:53:09.399199 7968 webserver.cc:460] Webserver started at http://127.0.191.195:38385/ using document root <none> and password file <none>
I20260926 06:53:09.399410 7968 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:09.399489 7968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:09.400791 7968 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:09.401322 7983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.401499 7968 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:09.401611 7968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "2c998e0fe06946d1ac61842d0d139930"
format_stamp: "Formatted at 2026-09-26 06:53:08 on dist-test-slave-ws74"
server_key: "5e55ee54a347445208c1ba6db476bfbb"
server_key_iv: "9284eeec500d21c1cec0e7fc5409f26d"
server_key_version: "encryptionkey@0"
I20260926 06:53:09.401937 7968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:09.423852 7968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:09.425700 7968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:09.425853 7968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:09.426074 7968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:09.426378 7968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:09.426430 7968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.426472 7968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:09.426499 7968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:09.432700 7968 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:41139
I20260926 06:53:09.432752 8096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:41139 every 8 connection(s)
I20260926 06:53:09.433064 7968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:53:09.439383 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 7968
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405589, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405589, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.252@KRBTEST.COM
Sep 26 06:53:09 dist-test-slave-ws74 krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405589, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.253@KRBTEST.COM
I20260926 06:53:09.445277 7830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.434039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:54793 (local address 127.0.191.252:34135)
0926 06:53:09.434093 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.434096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.434105 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:09.435196 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.435197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.435209 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.435242 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.436060 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.436814 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.437728 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.438009 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.438073 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.441586 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.441587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.441588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.441594 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.443473 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.443966 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.443968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.443968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.444027 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.444630 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.444631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.444631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.444773 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.444835 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.445078 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.445158 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 06:53:09.445318 8102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.433768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.195:57937)
0926 06:53:09.434097 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.434105 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.435074 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.435355 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.435359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.435823 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:09.435956 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.435961 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.436891 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.436892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.437763 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.437765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.438225 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.438226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.438291 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.438981 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:09.438986 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:09.441000 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:09.443832 (+ 2832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.443833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.443833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.444033 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.444229 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.444229 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.444230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.444344 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.444622 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:09.444627 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:09.444783 (+ 156us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.444904 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.445083 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":20,"thread_start_us":76,"threads_started":1}
I20260926 06:53:09.445585 8101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.433768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:34135 (local address 127.0.191.195:54793)
0926 06:53:09.433942 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.433984 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.435071 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.435355 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.435359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.435785 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:09.435956 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.435961 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.436905 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.436906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.437640 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.437642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.438092 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.438093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.438242 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.438927 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:09.438931 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:09.441271 (+ 2340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:09.443569 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.443575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.443577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.443891 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.444488 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.444489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.444490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.444555 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.444861 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:09.444861 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:09.444901 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.444989 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.445487 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":426,"thread_start_us":88,"threads_started":1}
I20260926 06:53:09.446002 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.434048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:57937 (local address 127.0.191.254:37691)
0926 06:53:09.434143 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.434144 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.434150 (+ 6us) server_negotiation.cc:407] Connection header received
0926 06:53:09.435236 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.435238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.435249 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.435298 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.436059 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.436808 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.437836 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.438128 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.438224 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.441146 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.441147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.441148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.441156 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.443748 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.444111 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.444112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.444113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.444169 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.444410 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.444411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.444411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.444543 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.444599 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.445835 (+ 1236us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.445916 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 06:53:09.446353 8098 heartbeater.cc:347] Connected to a master server at 127.0.191.254:37691
I20260926 06:53:09.446448 8098 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:09.446633 8098 heartbeater.cc:511] Master 127.0.191.254:37691 requested a full tablet report, sending...
I20260926 06:53:09.446973 6853 ts_manager.cc:194] Re-registered known tserver with Master: 2c998e0fe06946d1ac61842d0d139930 (127.0.191.195:41139)
I20260926 06:53:09.447125 7502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.434007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:35423 (local address 127.0.191.253:43773)
0926 06:53:09.434053 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.434056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.434396 (+ 340us) server_negotiation.cc:407] Connection header received
0926 06:53:09.435180 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.435181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.435193 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.435280 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.436046 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.436527 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.437842 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.438031 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.438092 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.442093 (+ 4001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.442095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.442095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.442103 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.443668 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.446337 (+ 2669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.446338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.446339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.446400 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.446600 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.446601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.446601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.446721 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.446788 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.446971 (+ 183us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.447045 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:53:09.447415 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.433768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:43773 (local address 127.0.191.195:35423)
0926 06:53:09.434327 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.434336 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.435078 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.435356 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.435359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.435769 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:09.435955 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.435959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.436626 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.436627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.437753 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.437755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.438164 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.438165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.438241 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.438902 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:09.438912 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:09.441996 (+ 3084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:09.443757 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.443758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.443758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.446246 (+ 2488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.446475 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:09.446475 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:09.446476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:09.446531 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:09.446820 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:09.446821 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:09.446861 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.447203 (+ 342us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.447306 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":26,"thread_start_us":75,"threads_started":1}
I20260926 06:53:09.447464 6853 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:57937
I20260926 06:53:09.447955 8099 heartbeater.cc:347] Connected to a master server at 127.0.191.253:43773
I20260926 06:53:09.448014 8099 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:09.448081 8099 heartbeater.cc:511] Master 127.0.191.253:43773 requested a full tablet report, sending...
I20260926 06:53:09.448688 8097 heartbeater.cc:347] Connected to a master server at 127.0.191.252:34135
I20260926 06:53:09.448760 8097 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:09.448829 8097 heartbeater.cc:511] Master 127.0.191.252:34135 requested a full tablet report, sending...
I20260926 06:53:09.448964 7451 ts_manager.cc:194] Registered new tserver with Master: 2c998e0fe06946d1ac61842d0d139930 (127.0.191.195:41139)
I20260926 06:53:09.449105 7645 ts_manager.cc:194] Registered new tserver with Master: 2c998e0fe06946d1ac61842d0d139930 (127.0.191.195:41139)
I20260926 06:53:09.452435 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.440836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40450 (local address 127.0.191.252:34135)
0926 06:53:09.441299 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.441301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.441308 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:53:09.441327 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.441328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.441340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.445030 (+ 3690us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.445060 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.446028 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.446998 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.447322 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.449193 (+ 1871us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.449643 (+ 450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.449645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.449645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.449651 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.451215 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.451545 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.451546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.451546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.451602 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.451837 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.451838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.451838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.451950 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.452049 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.452162 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.452237 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":57,"threads_started":1}
I20260926 06:53:09.452965 8113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.441463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58106 (local address 127.0.191.254:37691)
0926 06:53:09.441620 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.441622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.441629 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:53:09.441650 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.441650 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.441662 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.442620 (+ 958us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.442657 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.443454 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.444536 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.444820 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.445147 (+ 327us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.450090 (+ 4943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.450091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.450091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.450097 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.451736 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.452115 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.452116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.452116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.452179 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.452383 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.452384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.452384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.452514 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.452612 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.452789 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.452864 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260926 06:53:09.453491 8109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.440620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49448 (local address 127.0.191.253:43773)
0926 06:53:09.441870 (+ 1250us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.441872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.441879 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:53:09.441899 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.441900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.441911 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.445902 (+ 3991us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:09.445934 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.446678 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.447750 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.448058 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.448118 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.450362 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:09.450363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:09.450364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:09.450370 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:09.452245 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.452611 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.452612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.452613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.452669 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:09.452852 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:09.452853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:09.452853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:09.452980 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:09.453071 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.453282 (+ 211us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.453361 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1230,"thread_start_us":56,"threads_started":1}
I20260926 06:53:09.454547 6853 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:53:09.456023 6853 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:53:09.463542 7411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.459430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:41465 (local address 127.0.0.1:56192)
0926 06:53:09.459504 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.459517 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.459561 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.460124 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.460125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.460126 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:09.460155 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.460156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.461361 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.461362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.462435 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.462437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.463312 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.463313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.463369 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.463373 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.463438 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.463479 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":5}
I20260926 06:53:09.463776 8118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.459523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:37313 (local address 127.0.0.1:39666)
0926 06:53:09.459643 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.459652 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.459950 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.460275 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.460276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.460277 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:09.460300 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.460300 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.461297 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.461297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.462439 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.462441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.463513 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.463514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.463578 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.463582 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.463634 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.463685 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260926 06:53:09.463830 8119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.459556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56192 (local address 127.0.191.193:41465)
0926 06:53:09.459955 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.459959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.459970 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:53:09.460013 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.460015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.460032 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.460117 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:09.460203 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.461288 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.462510 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.463226 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.463313 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.463386 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.463452 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.463491 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":71,"threads_started":1}
I20260926 06:53:09.464033 8120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.459723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39666 (local address 127.0.191.194:37313)
0926 06:53:09.460101 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.460104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.460113 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:09.460149 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.460151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.460177 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.460256 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:09.460371 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.461191 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.462529 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.463346 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.463406 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.463473 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.463892 (+ 419us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.463941 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":109,"threads_started":1}
I20260926 06:53:09.464262 8121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.459800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:41139 (local address 127.0.0.1:57712)
0926 06:53:09.460335 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.460344 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.460392 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.460585 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.460585 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.460588 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:09.460626 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.460628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.461684 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.461685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.462540 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.462542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.464031 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.464032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.464095 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.464101 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.464159 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.464195 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":64,"threads_started":1}
I20260926 06:53:09.464457 8122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.459880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57712 (local address 127.0.191.195:41139)
0926 06:53:09.460028 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.460031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.460446 (+ 415us) server_negotiation.cc:407] Connection header received
0926 06:53:09.460484 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.460486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.460500 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.460569 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:09.460707 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.461607 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.462673 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.463335 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.463406 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.463933 (+ 527us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.464355 (+ 422us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.464393 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":63,"threads_started":1}
I20260926 06:53:09.464637 7895 tablet_service.cc:1467] Processing CreateTablet for tablet ab218b9942bc4d399b63d0afe77864f3 (DEFAULT_TABLE table=test-table [id=413e1ad35a754c47a4280db8d8efd69e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:09.464614 7758 tablet_service.cc:1467] Processing CreateTablet for tablet ab218b9942bc4d399b63d0afe77864f3 (DEFAULT_TABLE table=test-table [id=413e1ad35a754c47a4280db8d8efd69e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:09.464944 7895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab218b9942bc4d399b63d0afe77864f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:09.464963 7758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab218b9942bc4d399b63d0afe77864f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:09.465965 8031 tablet_service.cc:1467] Processing CreateTablet for tablet ab218b9942bc4d399b63d0afe77864f3 (DEFAULT_TABLE table=test-table [id=413e1ad35a754c47a4280db8d8efd69e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:09.466245 8031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab218b9942bc4d399b63d0afe77864f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:09.467634 8123 tablet_bootstrap.cc:492] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36: Bootstrap starting.
I20260926 06:53:09.468230 8123 tablet_bootstrap.cc:654] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:09.468297 8124 tablet_bootstrap.cc:492] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449: Bootstrap starting.
I20260926 06:53:09.468521 8123 log.cc:824] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:09.468892 8124 tablet_bootstrap.cc:654] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:09.469362 8124 log.cc:824] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:09.469733 8123 tablet_bootstrap.cc:492] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36: No bootstrap required, opened a new log
I20260926 06:53:09.469776 8123 ts_tablet_manager.cc:1397] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:53:09.469995 8127 tablet_bootstrap.cc:492] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930: Bootstrap starting.
I20260926 06:53:09.470165 8124 tablet_bootstrap.cc:492] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449: No bootstrap required, opened a new log
I20260926 06:53:09.470227 8124 ts_tablet_manager.cc:1397] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:53:09.470580 8127 tablet_bootstrap.cc:654] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:09.470850 8127 log.cc:824] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:09.471577 8127 tablet_bootstrap.cc:492] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930: No bootstrap required, opened a new log
I20260926 06:53:09.471642 8127 ts_tablet_manager.cc:1397] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:53:09.472110 8123 raft_consensus.cc:348] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:09.472270 8123 raft_consensus.cc:374] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:09.472306 8123 raft_consensus.cc:729] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8e80e1b50742878a46ee35b9e33c36, State: Initialized, Role: FOLLOWER
I20260926 06:53:09.472299 8124 raft_consensus.cc:348] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:09.472447 8123 consensus_queue.cc:260] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:09.472491 8124 raft_consensus.cc:374] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:09.472538 8124 raft_consensus.cc:729] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 888b433ce9094d4295a686cebda88449, State: Initialized, Role: FOLLOWER
I20260926 06:53:09.472659 8124 consensus_queue.cc:260] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:09.472702 8123 ts_tablet_manager.cc:1428] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:53:09.472903 8124 ts_tablet_manager.cc:1428] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:53:09.473017 7962 heartbeater.cc:503] Master 127.0.191.254:37691 was elected leader, sending a full tablet report...
I20260926 06:53:09.473557 8127 raft_consensus.cc:348] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:09.473701 8127 raft_consensus.cc:374] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:09.473733 8127 raft_consensus.cc:729] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c998e0fe06946d1ac61842d0d139930, State: Initialized, Role: FOLLOWER
I20260926 06:53:09.473839 8127 consensus_queue.cc:260] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:09.474073 8127 ts_tablet_manager.cc:1428] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 06:53:09.474079 7825 heartbeater.cc:503] Master 127.0.191.254:37691 was elected leader, sending a full tablet report...
I20260926 06:53:09.474154 8098 heartbeater.cc:503] Master 127.0.191.254:37691 was elected leader, sending a full tablet report...
W20260926 06:53:09.541849 7964 tablet.cc:2369] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:53:09.622395 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.620139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40454 (local address 127.0.191.252:34135)
0926 06:53:09.620205 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.620207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.620223 (+ 16us) server_negotiation.cc:407] Connection header received
0926 06:53:09.620250 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.620251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.620269 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.620338 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:09.620459 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.620946 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.621785 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.622157 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.622216 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.622268 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.622316 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.622339 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 06:53:09.622403 8121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.620039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:34135 (local address 127.0.0.1:40454)
0926 06:53:09.620103 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.620120 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.620157 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.620353 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.620354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.620355 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:09.620399 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.620401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.621007 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.621007 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.621708 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.621710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.622226 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.622227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.622271 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.622274 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.622312 (+ 38us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.622349 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 06:53:09.622777 7655 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260926 06:53:09.623240 8132 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 957ffa2a68c546fe923af9f52bc212bb. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } } }
I20260926 06:53:09.623368 8132 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: This master's current role is: LEARNER
W20260926 06:53:09.660737 7827 tablet.cc:2369] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:53:09.684653 8100 tablet.cc:2369] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:53:09.900772 8131 raft_consensus.cc:482] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:53:09.900920 8131 raft_consensus.cc:504] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:09.901320 8131 leader_election.cc:288] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a8e80e1b50742878a46ee35b9e33c36 (127.0.191.193:41465), 888b433ce9094d4295a686cebda88449 (127.0.191.194:37313)
I20260926 06:53:09.904055 8120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.901653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:42265 (local address 127.0.191.194:37313)
0926 06:53:09.901708 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.901709 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.901718 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:09.901742 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.901743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.901751 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.901802 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:09.901938 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.902400 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.903354 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.903714 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.903800 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.903809 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.903952 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.903986 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:53:09.904116 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.901491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:41465 (local address 127.0.191.195:46519)
0926 06:53:09.901554 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.901568 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.901622 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.901783 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.901784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.901785 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:09.901846 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.901847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.902482 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.902483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.903281 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.903283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.903841 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.903842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.903921 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.903924 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.903997 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.904037 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":29}
I20260926 06:53:09.904110 8119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.901567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:46519 (local address 127.0.191.193:41465)
0926 06:53:09.901624 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:09.901626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:09.901636 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:53:09.901699 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:09.901699 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:09.901709 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:09.901779 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:09.901924 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.902394 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.903370 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.903733 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.903842 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.903849 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:09.904005 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 06:53:09.904043 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:53:09.904230 8101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:09.901587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:37313 (local address 127.0.191.195:42265)
0926 06:53:09.901627 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:09.901638 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:09.901673 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:09.901803 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:09.901803 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:09.901805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:09.901867 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.901868 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.902480 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.902481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.903259 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:09.903261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:09.903797 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:09.903798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:09.903871 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:09.903874 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:09.903948 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 06:53:09.903986 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260926 06:53:09.904758 7915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab218b9942bc4d399b63d0afe77864f3" candidate_uuid: "2c998e0fe06946d1ac61842d0d139930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "888b433ce9094d4295a686cebda88449" is_pre_election: true
I20260926 06:53:09.904779 7778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab218b9942bc4d399b63d0afe77864f3" candidate_uuid: "2c998e0fe06946d1ac61842d0d139930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8e80e1b50742878a46ee35b9e33c36" is_pre_election: true
I20260926 06:53:09.904919 7915 raft_consensus.cc:2454] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c998e0fe06946d1ac61842d0d139930 in term 0.
I20260926 06:53:09.904937 7778 raft_consensus.cc:2454] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c998e0fe06946d1ac61842d0d139930 in term 0.
I20260926 06:53:09.905143 7987 leader_election.cc:302] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c998e0fe06946d1ac61842d0d139930, 888b433ce9094d4295a686cebda88449; no voters:
I20260926 06:53:09.905278 8131 raft_consensus.cc:2781] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:53:09.905360 8131 raft_consensus.cc:482] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:53:09.905390 8131 raft_consensus.cc:3037] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:09.906118 8131 raft_consensus.cc:504] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:09.906263 8131 leader_election.cc:288] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [CANDIDATE]: Term 1 election: Requested vote from peers 6a8e80e1b50742878a46ee35b9e33c36 (127.0.191.193:41465), 888b433ce9094d4295a686cebda88449 (127.0.191.194:37313)
I20260926 06:53:09.906422 7915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab218b9942bc4d399b63d0afe77864f3" candidate_uuid: "2c998e0fe06946d1ac61842d0d139930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "888b433ce9094d4295a686cebda88449"
I20260926 06:53:09.906437 7778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab218b9942bc4d399b63d0afe77864f3" candidate_uuid: "2c998e0fe06946d1ac61842d0d139930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8e80e1b50742878a46ee35b9e33c36"
I20260926 06:53:09.906502 7915 raft_consensus.cc:3037] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:09.906530 7778 raft_consensus.cc:3037] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:09.907172 7778 raft_consensus.cc:2454] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c998e0fe06946d1ac61842d0d139930 in term 1.
I20260926 06:53:09.907191 7915 raft_consensus.cc:2454] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c998e0fe06946d1ac61842d0d139930 in term 1.
I20260926 06:53:09.907327 7987 leader_election.cc:302] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c998e0fe06946d1ac61842d0d139930, 6a8e80e1b50742878a46ee35b9e33c36; no voters:
I20260926 06:53:09.907409 8131 raft_consensus.cc:2781] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:09.907518 8131 raft_consensus.cc:686] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [term 1 LEADER]: Becoming Leader. State: Replica: 2c998e0fe06946d1ac61842d0d139930, State: Running, Role: LEADER
I20260926 06:53:09.907619 8131 consensus_queue.cc:237] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:09.908393 6853 catalog_manager.cc:5495] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c998e0fe06946d1ac61842d0d139930 (127.0.191.195). New cstate: current_term: 1 leader_uuid: "2c998e0fe06946d1ac61842d0d139930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } health_report { overall_health: UNKNOWN } } }
I20260926 06:53:09.908952 7609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: NON_VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: true } } }
I20260926 06:53:09.909046 7609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:09.909739 8138 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb: attempting to promote NON_VOTER f40b9326c65f4616a70fbf25924e152e to VOTER
I20260926 06:53:09.909896 8138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } }
I20260926 06:53:09.910310 7655 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 LEARNER]: Refusing update from remote peer 957ffa2a68c546fe923af9f52bc212bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260926 06:53:09.910356 7461 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 FOLLOWER]: Refusing update from remote peer 957ffa2a68c546fe923af9f52bc212bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260926 06:53:09.910640 8139 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260926 06:53:09.910727 7609 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260926 06:53:09.911262 8139 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f40b9326c65f4616a70fbf25924e152e (127.0.191.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } } }
I20260926 06:53:09.911468 7655 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f40b9326c65f4616a70fbf25924e152e (127.0.191.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } } }
I20260926 06:53:09.911489 7461 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f40b9326c65f4616a70fbf25924e152e (127.0.191.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } } }
I20260926 06:53:09.912153 8139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } } }
I20260926 06:53:09.912252 8139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:09.912313 8139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } } }
I20260926 06:53:09.912376 8139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:09.912364 8141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } } }
I20260926 06:53:09.912429 8139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } } }
I20260926 06:53:09.912451 8141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:53:09.912467 8139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:09.912554 8132 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "957ffa2a68c546fe923af9f52bc212bb" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } } }
I20260926 06:53:09.912623 8132 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:53:10.013723 8144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:10.011156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57728 (local address 127.0.191.195:41139)
0926 06:53:10.011269 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:10.011271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:10.011281 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:53:10.011325 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:10.011325 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:10.011347 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:10.011446 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:53:10.011584 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:10.012079 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:10.012837 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:10.013056 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:10.013142 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:10.013317 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:53:10.013492 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:53:10.013589 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:10.013608 (+ 19us) server_negotiation.cc:299] Negotiation successful
0926 06:53:10.013629 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":46,"threads_started":1}
I20260926 06:53:10.015475 7915 raft_consensus.cc:1261] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449 [term 1 FOLLOWER]: Refusing update from remote peer 2c998e0fe06946d1ac61842d0d139930: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:10.015494 7778 raft_consensus.cc:1261] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36 [term 1 FOLLOWER]: Refusing update from remote peer 2c998e0fe06946d1ac61842d0d139930: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:10.015851 8131 consensus_queue.cc:1035] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:10.015990 8136 consensus_queue.cc:1035] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:10.018968 8145 mvcc.cc:205] Tried to move back new op lower bound from 7333501296700624896 to 7333501296262201344. Current Snapshot: MvccSnapshot[applied={T|T < 7333501296700624896 or (T in {7333501296700624896})}]
I20260926 06:53:10.019069 8149 mvcc.cc:205] Tried to move back new op lower bound from 7333501296700624896 to 7333501296262201344. Current Snapshot: MvccSnapshot[applied={T|T < 7333501296700624896 or (T in {7333501296700624896})}]
I20260926 06:53:10.020629 8148 mvcc.cc:205] Tried to move back new op lower bound from 7333501296700624896 to 7333501296262201344. Current Snapshot: MvccSnapshot[applied={T|T < 7333501296700624896}]
I20260926 06:53:10.020677 6851 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:10.020766 6851 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:10.022164 6851 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 957ffa2a68c546fe923af9f52bc212bb: Sending DeleteTablet for 3 replicas of tablet ab218b9942bc4d399b63d0afe77864f3
I20260926 06:53:10.022476 7895 tablet_service.cc:1514] Processing DeleteTablet for tablet ab218b9942bc4d399b63d0afe77864f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:10 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:39666
I20260926 06:53:10.022743 8156 tablet_replica.cc:330] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449: stopping tablet replica
I20260926 06:53:10.022833 8156 raft_consensus.cc:2229] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:10.022853 7758 tablet_service.cc:1514] Processing DeleteTablet for tablet ab218b9942bc4d399b63d0afe77864f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:10 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:56192
I20260926 06:53:10.022923 8156 raft_consensus.cc:2258] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:10.023283 8031 tablet_service.cc:1514] Processing DeleteTablet for tablet ab218b9942bc4d399b63d0afe77864f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:10 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:57712
I20260926 06:53:10.023317 8156 ts_tablet_manager.cc:1905] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:10.023592 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 6821
I20260926 06:53:10.023692 8158 tablet_replica.cc:330] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36: stopping tablet replica
I20260926 06:53:10.023779 8158 raft_consensus.cc:2229] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:10.023866 8158 raft_consensus.cc:2258] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:10.024013 8159 tablet_replica.cc:330] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930: stopping tablet replica
I20260926 06:53:10.024091 8159 raft_consensus.cc:2229] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:53:10.024196 8159 raft_consensus.cc:2258] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:10.024531 8159 ts_tablet_manager.cc:1905] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:10.025513 8156 ts_tablet_manager.cc:1918] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:10.025576 8156 log.cc:1196] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/ab218b9942bc4d399b63d0afe77864f3
I20260926 06:53:10.025835 8156 ts_tablet_manager.cc:1939] T ab218b9942bc4d399b63d0afe77864f3 P 888b433ce9094d4295a686cebda88449: Deleting consensus metadata
I20260926 06:53:10.026381 8159 ts_tablet_manager.cc:1918] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:10.026446 8159 log.cc:1196] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/ab218b9942bc4d399b63d0afe77864f3
I20260926 06:53:10.026682 8159 ts_tablet_manager.cc:1939] T ab218b9942bc4d399b63d0afe77864f3 P 2c998e0fe06946d1ac61842d0d139930: Deleting consensus metadata
I20260926 06:53:10.027206 8158 ts_tablet_manager.cc:1905] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:10.028980 8158 ts_tablet_manager.cc:1918] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:10.029042 8158 log.cc:1196] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-0/wal/wals/ab218b9942bc4d399b63d0afe77864f3
I20260926 06:53:10.029281 8158 ts_tablet_manager.cc:1939] T ab218b9942bc4d399b63d0afe77864f3 P 6a8e80e1b50742878a46ee35b9e33c36: Deleting consensus metadata
W20260926 06:53:10.031044 7987 connection.cc:535] server connection from 127.0.0.1:57712 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 06:53:10.031358 7850 connection.cc:535] server connection from 127.0.0.1:39666 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 06:53:10.031572 7714 connection.cc:535] server connection from 127.0.0.1:56192 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 06:53:10.033911 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:10.023081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40466 (local address 127.0.191.252:34135)
0926 06:53:10.023123 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:10.023125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:10.025878 (+ 2753us) server_negotiation.cc:407] Connection header received
0926 06:53:10.025902 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:10.025903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:10.025914 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:10.025978 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:10.026705 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:10.027488 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:10.028537 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:10.028844 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:10.028889 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:10.030870 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:10.030871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:10.030872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:10.030879 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:10.032639 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:10.033031 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:10.033032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:10.033032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:10.033085 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:10.033293 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:10.033293 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:10.033294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:10.033445 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:10.033518 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:10.033738 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 06:53:10.033816 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:53:10.040521 8165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:10.033246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49450 (local address 127.0.191.253:43773)
0926 06:53:10.033343 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:10.033346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:10.033354 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:53:10.033396 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:10.033397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:10.033409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:10.034048 (+ 639us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:10.034086 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:10.034876 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:10.035882 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:10.036202 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:10.036288 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:10.037596 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:10.037597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:10.037598 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:10.037604 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:10.039247 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:10.039682 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:10.039683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:10.039684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:10.039748 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:10.039948 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:10.039949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:10.039949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:10.040082 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:10.040174 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:10.040335 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 06:53:10.040419 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":56,"threads_started":1}
I20260926 06:53:10.041004 8166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:10.033492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40468 (local address 127.0.191.252:34135)
0926 06:53:10.033718 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:10.033720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:10.033882 (+ 162us) server_negotiation.cc:407] Connection header received
0926 06:53:10.033901 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:10.033902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:10.033912 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:10.034196 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:10.034852 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:10.035351 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:10.036146 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:10.036464 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:10.036525 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:10.037839 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:10.037840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:10.037841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:10.037847 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:10.039442 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:10.039950 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:10.039951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:10.039951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:10.039996 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:10.040186 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:10.040187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:10.040187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:10.040314 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:10.040400 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:10.040801 (+ 401us) server_negotiation.cc:299] Negotiation successful
0926 06:53:10.040907 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":52,"threads_started":1}
I20260926 06:53:11.020071 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.019777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.195:55381)
0926 06:53:11.019907 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:11.020014 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260926 06:53:11.020116 8169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.019875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.193:44157)
0926 06:53:11.019986 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:11.020058 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":52,"threads_started":1}
W20260926 06:53:11.020485 8098 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:37691 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
W20260926 06:53:11.020582 7825 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:37691 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
I20260926 06:53:11.023568 8170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.023396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.194:57643)
0926 06:53:11.023484 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:11.023535 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":42,"threads_started":1}
W20260926 06:53:11.023746 7962 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:37691 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
I20260926 06:53:11.523092 8171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 957ffa2a68c546fe923af9f52bc212bb)
I20260926 06:53:11.523244 8171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } }
I20260926 06:53:11.523383 8172 raft_consensus.cc:482] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 957ffa2a68c546fe923af9f52bc212bb)
I20260926 06:53:11.523550 8171 leader_election.cc:288] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 957ffa2a68c546fe923af9f52bc212bb (127.0.191.254:37691), f40b9326c65f4616a70fbf25924e152e (127.0.191.252:34135)
I20260926 06:53:11.523500 8172 raft_consensus.cc:504] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } }
I20260926 06:53:11.523963 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.523644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59462)
0926 06:53:11.523859 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:11.523904 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":64,"threads_started":1}
I20260926 06:53:11.524065 8175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.523889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59466)
0926 06:53:11.523988 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:11.524033 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":48,"threads_started":1}
W20260926 06:53:11.524191 7435 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
W20260926 06:53:11.524219 7629 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
I20260926 06:53:11.524339 8172 leader_election.cc:288] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 957ffa2a68c546fe923af9f52bc212bb (127.0.191.254:37691), 362c18ce7b844446b05453009f59c391 (127.0.191.253:43773)
I20260926 06:53:11.524732 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.524617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59478)
0926 06:53:11.524674 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:11.524705 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260926 06:53:11.524822 7435 leader_election.cc:334] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 957ffa2a68c546fe923af9f52bc212bb (127.0.191.254:37691): Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
I20260926 06:53:11.524845 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.524620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59486)
0926 06:53:11.524775 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:11.524809 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
W20260926 06:53:11.524897 7629 leader_election.cc:334] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 957ffa2a68c546fe923af9f52bc212bb (127.0.191.254:37691): Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
I20260926 06:53:11.526919 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.523677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:34135 (local address 127.0.0.1:55314)
0926 06:53:11.523931 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:11.523944 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:11.523999 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:11.524341 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:11.524342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:11.524345 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:11.524407 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:11.524410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:11.525282 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:11.525283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:11.526101 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:11.526103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:11.526670 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:11.526671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:11.526732 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:11.526736 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:53:11.526813 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 06:53:11.526856 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":65,"threads_started":1}
I20260926 06:53:11.526917 8176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.523969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55314 (local address 127.0.191.252:34135)
0926 06:53:11.524164 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:11.524167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:11.524176 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:11.524200 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:11.524201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:11.524212 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:11.524287 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:11.524485 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:11.525206 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:11.526199 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:11.526576 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:11.526667 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:11.526677 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:11.526820 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 06:53:11.526857 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":39,"threads_started":1}
I20260926 06:53:11.527238 8175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.524134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:43773 (local address 127.0.0.1:33838)
0926 06:53:11.524210 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:11.524220 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:11.524269 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:11.524949 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:11.524950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:11.524953 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:11.524990 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:11.524991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:11.525702 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:11.525703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:11.526427 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:11.526429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:11.527031 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:11.527032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:11.527096 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:11.527100 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:53:11.527130 (+ 30us) client_negotiation.cc:240] Negotiation successful
0926 06:53:11.527166 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 06:53:11.527279 7655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "362c18ce7b844446b05453009f59c391" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f40b9326c65f4616a70fbf25924e152e" is_pre_election: true
I20260926 06:53:11.527385 7655 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 362c18ce7b844446b05453009f59c391 in term 1.
I20260926 06:53:11.527395 8179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.524365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.0.191.253:43773)
0926 06:53:11.524689 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:11.524691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:11.524700 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:11.524721 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:11.524722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:11.524731 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:11.524922 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:11.525053 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:11.525593 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:11.526529 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:11.526886 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:11.527241 (+ 355us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:11.527254 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:11.527273 (+ 19us) server_negotiation.cc:299] Negotiation successful
0926 06:53:11.527317 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":92,"threads_started":1}
I20260926 06:53:11.527537 7438 leader_election.cc:302] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 362c18ce7b844446b05453009f59c391, f40b9326c65f4616a70fbf25924e152e; no voters: 957ffa2a68c546fe923af9f52bc212bb
I20260926 06:53:11.527549 7461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f40b9326c65f4616a70fbf25924e152e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "362c18ce7b844446b05453009f59c391" is_pre_election: true
I20260926 06:53:11.527640 8171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 06:53:11.527683 7461 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f40b9326c65f4616a70fbf25924e152e in term 1.
I20260926 06:53:11.527710 8171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 957ffa2a68c546fe923af9f52bc212bb)
I20260926 06:53:11.527726 8171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 1 FOLLOWER]: Advancing to term 2
I20260926 06:53:11.527851 7630 leader_election.cc:302] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 362c18ce7b844446b05453009f59c391, f40b9326c65f4616a70fbf25924e152e; no voters: 957ffa2a68c546fe923af9f52bc212bb
I20260926 06:53:11.527963 8172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 06:53:11.528007 8172 raft_consensus.cc:482] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 957ffa2a68c546fe923af9f52bc212bb)
I20260926 06:53:11.528038 8172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 1 FOLLOWER]: Advancing to term 2
I20260926 06:53:11.528512 8171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } }
I20260926 06:53:11.528649 8171 leader_election.cc:288] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [CANDIDATE]: Term 2 election: Requested vote from peers 957ffa2a68c546fe923af9f52bc212bb (127.0.191.254:37691), f40b9326c65f4616a70fbf25924e152e (127.0.191.252:34135)
I20260926 06:53:11.528710 8172 raft_consensus.cc:504] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } }
I20260926 06:53:11.528781 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.528710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59494)
0926 06:53:11.528738 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:11.528762 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260926 06:53:11.528864 8172 leader_election.cc:288] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [CANDIDATE]: Term 2 election: Requested vote from peers 957ffa2a68c546fe923af9f52bc212bb (127.0.191.254:37691), 362c18ce7b844446b05453009f59c391 (127.0.191.253:43773)
I20260926 06:53:11.529074 8175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.528986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59498)
0926 06:53:11.529019 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:11.529051 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260926 06:53:11.529080 7461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f40b9326c65f4616a70fbf25924e152e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "362c18ce7b844446b05453009f59c391"
I20260926 06:53:11.529143 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.529037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59514)
0926 06:53:11.529089 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:11.529118 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 06:53:11.529151 7655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "362c18ce7b844446b05453009f59c391" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f40b9326c65f4616a70fbf25924e152e"
I20260926 06:53:11.529219 7655 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 362c18ce7b844446b05453009f59c391 in current term 2: Already voted for candidate f40b9326c65f4616a70fbf25924e152e in this term.
I20260926 06:53:11.529160 7461 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f40b9326c65f4616a70fbf25924e152e in current term 2: Already voted for candidate 362c18ce7b844446b05453009f59c391 in this term.
W20260926 06:53:11.529537 7435 leader_election.cc:334] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 957ffa2a68c546fe923af9f52bc212bb (127.0.191.254:37691): Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
I20260926 06:53:11.529573 8175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:11.529467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59528)
0926 06:53:11.529524 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:11.529552 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 06:53:11.529608 7435 leader_election.cc:302] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 362c18ce7b844446b05453009f59c391; no voters: 957ffa2a68c546fe923af9f52bc212bb, f40b9326c65f4616a70fbf25924e152e
W20260926 06:53:11.529618 7629 leader_election.cc:334] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 957ffa2a68c546fe923af9f52bc212bb (127.0.191.254:37691): Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
I20260926 06:53:11.529647 7629 leader_election.cc:302] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f40b9326c65f4616a70fbf25924e152e; no voters: 362c18ce7b844446b05453009f59c391, 957ffa2a68c546fe923af9f52bc212bb
I20260926 06:53:11.529663 8171 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 06:53:11.529712 8172 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 06:53:12.021265 8181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:12.021008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.193:58471)
0926 06:53:12.021145 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:12.021191 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260926 06:53:12.021512 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:12.021008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.195:58239)
0926 06:53:12.021436 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:12.021470 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":61,"threads_started":1}
I20260926 06:53:12.024194 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:12.024004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.194:47379)
0926 06:53:12.024120 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:12.024154 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260926 06:53:13.022110 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.021887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.193:35419)
0926 06:53:13.022011 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.022052 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260926 06:53:13.022236 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.021922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.195:42869)
0926 06:53:13.022139 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.022175 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":53,"threads_started":1}
I20260926 06:53:13.024698 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.024562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.194:39437)
0926 06:53:13.024643 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.024665 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69,"thread_start_us":39,"threads_started":1}
I20260926 06:53:13.557049 8187 raft_consensus.cc:482] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:53:13.557140 8187 raft_consensus.cc:504] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } }
I20260926 06:53:13.557351 8187 leader_election.cc:288] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 957ffa2a68c546fe923af9f52bc212bb (127.0.191.254:37691), 362c18ce7b844446b05453009f59c391 (127.0.191.253:43773)
I20260926 06:53:13.557638 7461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f40b9326c65f4616a70fbf25924e152e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "362c18ce7b844446b05453009f59c391" is_pre_election: true
I20260926 06:53:13.557703 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.557504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59824)
0926 06:53:13.557621 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.557668 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":40,"threads_started":1}
I20260926 06:53:13.557744 7461 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f40b9326c65f4616a70fbf25924e152e in term 2.
I20260926 06:53:13.557906 7630 leader_election.cc:302] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 362c18ce7b844446b05453009f59c391, f40b9326c65f4616a70fbf25924e152e; no voters:
I20260926 06:53:13.558004 8187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 06:53:13.558039 8187 raft_consensus.cc:482] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:53:13.558060 8187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 2 FOLLOWER]: Advancing to term 3
I20260926 06:53:13.558103 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.557994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59836)
0926 06:53:13.558027 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.558058 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260926 06:53:13.558167 7629 leader_election.cc:334] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 957ffa2a68c546fe923af9f52bc212bb (127.0.191.254:37691): Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
I20260926 06:53:13.558883 8187 raft_consensus.cc:504] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } }
I20260926 06:53:13.559017 8187 leader_election.cc:288] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [CANDIDATE]: Term 3 election: Requested vote from peers 957ffa2a68c546fe923af9f52bc212bb (127.0.191.254:37691), 362c18ce7b844446b05453009f59c391 (127.0.191.253:43773)
I20260926 06:53:13.559207 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.559119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59844)
0926 06:53:13.559152 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.559184 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260926 06:53:13.559163 7461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f40b9326c65f4616a70fbf25924e152e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "362c18ce7b844446b05453009f59c391"
I20260926 06:53:13.559276 7461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 2 FOLLOWER]: Advancing to term 3
I20260926 06:53:13.559552 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.559463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59846)
0926 06:53:13.559508 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.559534 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 06:53:13.559628 7629 leader_election.cc:334] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 957ffa2a68c546fe923af9f52bc212bb (127.0.191.254:37691): Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
I20260926 06:53:13.559985 7461 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f40b9326c65f4616a70fbf25924e152e in term 3.
I20260926 06:53:13.560146 7630 leader_election.cc:302] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 362c18ce7b844446b05453009f59c391, f40b9326c65f4616a70fbf25924e152e; no voters: 957ffa2a68c546fe923af9f52bc212bb
I20260926 06:53:13.560276 8187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 3 FOLLOWER]: Leader election won for term 3
I20260926 06:53:13.560359 8187 raft_consensus.cc:686] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [term 3 LEADER]: Becoming Leader. State: Replica: f40b9326c65f4616a70fbf25924e152e, State: Running, Role: LEADER
I20260926 06:53:13.560411 8187 consensus_queue.cc:237] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } }
I20260926 06:53:13.560719 8190 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f40b9326c65f4616a70fbf25924e152e. Latest consensus state: current_term: 3 leader_uuid: "f40b9326c65f4616a70fbf25924e152e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } } }
I20260926 06:53:13.560817 8190 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:13.560979 8192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:13.561357 8192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:13.561468 8192 catalog_manager.cc:1259] Loaded cluster ID: 9145caa6548345cb96badc48689bd591
I20260926 06:53:13.561501 8192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:53:13.561661 8192 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:13.561739 8192 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: Loaded TSK: 0
I20260926 06:53:13.561842 8192 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:53:13.611518 7645 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:53:13.613049 7645 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:53:13.614095 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.613971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59872)
0926 06:53:13.614025 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.614062 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 06:53:13.614074 7461 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [term 3 FOLLOWER]: Refusing update from remote peer f40b9326c65f4616a70fbf25924e152e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260926 06:53:13.614310 8190 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [LEADER]: Connected to new peer: Peer: permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260926 06:53:13.614405 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.614323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.0.1:59876)
0926 06:53:13.614354 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.614380 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260926 06:53:13.614470 7629 consensus_peers.cc:487] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e -> Peer 957ffa2a68c546fe923af9f52bc212bb (127.0.191.254:37691): Couldn't send request to peer 957ffa2a68c546fe923af9f52bc212bb. Status: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 06:53:13.614938 8194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f40b9326c65f4616a70fbf25924e152e. Latest consensus state: current_term: 3 leader_uuid: "f40b9326c65f4616a70fbf25924e152e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } } }
I20260926 06:53:13.615048 8194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:53:13.615470 8190 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f40b9326c65f4616a70fbf25924e152e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } } }
I20260926 06:53:13.615561 8190 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:13.615675 8194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f40b9326c65f4616a70fbf25924e152e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "957ffa2a68c546fe923af9f52bc212bb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37691 } } peers { permanent_uuid: "362c18ce7b844446b05453009f59c391" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 43773 } attrs { promote: false } } peers { permanent_uuid: "f40b9326c65f4616a70fbf25924e152e" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 34135 } attrs { promote: false } } }
I20260926 06:53:13.615774 8194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362c18ce7b844446b05453009f59c391 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:53:13.620304 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.617459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34984 (local address 127.0.191.194:37313)
0926 06:53:13.617609 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:13.617611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:13.617620 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:13.617644 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:13.617645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:13.617717 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:13.617780 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:13.617934 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.618601 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.619515 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.619975 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.620055 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:13.620066 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:13.620198 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 06:53:13.620237 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":61,"threads_started":1}
I20260926 06:53:13.620304 8201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.617204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:37313 (local address 127.0.0.1:34984)
0926 06:53:13.617351 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.617362 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:13.617432 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:13.617820 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:13.617821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:13.617823 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:13.617861 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:13.617862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.618680 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.618681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:13.619389 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:13.619391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.620053 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.620054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:13.620114 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:13.620118 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:53:13.620190 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:53:13.620231 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":67,"threads_started":1}
I20260926 06:53:13.620560 8200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.617104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:41139 (local address 127.0.0.1:48546)
0926 06:53:13.617273 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.617282 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:13.617322 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:13.617581 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:13.617582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:13.617584 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:13.617636 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:13.617637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.618367 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.618368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:13.619568 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:13.619570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.620331 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.620332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:13.620402 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:13.620404 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:53:13.620434 (+ 30us) client_negotiation.cc:240] Negotiation successful
0926 06:53:13.620481 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":66,"thread_start_us":63,"threads_started":1}
I20260926 06:53:13.620690 8202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.617252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42476 (local address 127.0.191.193:41465)
0926 06:53:13.617350 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:13.617352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:13.617362 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:53:13.617411 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:13.617412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:13.617424 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:13.617513 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:13.617644 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.618421 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.619668 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.620240 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.620524 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:13.620538 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:13.620557 (+ 19us) server_negotiation.cc:299] Negotiation successful
0926 06:53:13.620605 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":48,"threads_started":1}
I20260926 06:53:13.620833 7895 tablet_service.cc:1467] Processing CreateTablet for tablet d6fccc665cad4550b78597e4c128b3e4 (DEFAULT_TABLE table=test-table [id=5a4bc9a83e5e4b0182bd5156bd09e9ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:13.620921 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.617104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:41465 (local address 127.0.0.1:42476)
0926 06:53:13.617165 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.617175 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:13.617235 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:13.617515 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:13.617516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:13.617517 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:13.617567 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:13.617568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.618495 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.618496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:13.619570 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:13.619571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.620332 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.620332 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:13.620399 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:13.620403 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:53:13.620434 (+ 31us) client_negotiation.cc:240] Negotiation successful
0926 06:53:13.620481 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":5}
I20260926 06:53:13.620980 7895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fccc665cad4550b78597e4c128b3e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:13.621196 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.617321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48546 (local address 127.0.191.195:41139)
0926 06:53:13.617444 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:13.617447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:13.617454 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:53:13.617478 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:13.617479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:13.617491 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:13.617700 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:13.617754 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.618289 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.619669 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.620233 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.621041 (+ 808us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:13.621058 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:13.621084 (+ 26us) server_negotiation.cc:299] Negotiation successful
0926 06:53:13.621125 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":86,"threads_started":1}
I20260926 06:53:13.621248 7758 tablet_service.cc:1467] Processing CreateTablet for tablet d6fccc665cad4550b78597e4c128b3e4 (DEFAULT_TABLE table=test-table [id=5a4bc9a83e5e4b0182bd5156bd09e9ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:13.621393 7758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fccc665cad4550b78597e4c128b3e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:13.621536 8031 tablet_service.cc:1467] Processing CreateTablet for tablet d6fccc665cad4550b78597e4c128b3e4 (DEFAULT_TABLE table=test-table [id=5a4bc9a83e5e4b0182bd5156bd09e9ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:13.621637 8031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fccc665cad4550b78597e4c128b3e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:13.622407 8205 tablet_bootstrap.cc:492] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449: Bootstrap starting.
I20260926 06:53:13.622728 8205 tablet_bootstrap.cc:654] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:13.622825 8206 tablet_bootstrap.cc:492] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36: Bootstrap starting.
I20260926 06:53:13.623041 8207 tablet_bootstrap.cc:492] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930: Bootstrap starting.
I20260926 06:53:13.623178 8206 tablet_bootstrap.cc:654] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:13.623432 8207 tablet_bootstrap.cc:654] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:13.623652 8205 tablet_bootstrap.cc:492] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449: No bootstrap required, opened a new log
I20260926 06:53:13.623709 8205 ts_tablet_manager.cc:1397] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:13.623853 8205 raft_consensus.cc:348] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:13.623989 8207 tablet_bootstrap.cc:492] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930: No bootstrap required, opened a new log
I20260926 06:53:13.624001 8206 tablet_bootstrap.cc:492] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36: No bootstrap required, opened a new log
I20260926 06:53:13.624015 8205 raft_consensus.cc:374] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:13.624044 8205 raft_consensus.cc:729] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 888b433ce9094d4295a686cebda88449, State: Initialized, Role: FOLLOWER
I20260926 06:53:13.624043 8207 ts_tablet_manager.cc:1397] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:13.624058 8206 ts_tablet_manager.cc:1397] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:13.624110 8205 consensus_queue.cc:260] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:13.624142 8207 raft_consensus.cc:348] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:13.624207 8207 raft_consensus.cc:374] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:13.624231 8207 raft_consensus.cc:729] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c998e0fe06946d1ac61842d0d139930, State: Initialized, Role: FOLLOWER
I20260926 06:53:13.624277 8207 consensus_queue.cc:260] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:13.624374 8205 ts_tablet_manager.cc:1428] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:13.624399 8207 ts_tablet_manager.cc:1428] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:13.624233 8206 raft_consensus.cc:348] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:13.624874 8206 raft_consensus.cc:374] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:13.624940 8206 raft_consensus.cc:729] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8e80e1b50742878a46ee35b9e33c36, State: Initialized, Role: FOLLOWER
I20260926 06:53:13.624994 8206 consensus_queue.cc:260] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:13.625217 8206 ts_tablet_manager.cc:1428] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 06:53:13.625473 7961 heartbeater.cc:503] Master 127.0.191.252:34135 was elected leader, sending a full tablet report...
I20260926 06:53:13.625648 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.624476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.194:58815)
0926 06:53:13.625572 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.625610 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1074,"thread_start_us":72,"threads_started":1}
I20260926 06:53:13.625849 7824 heartbeater.cc:503] Master 127.0.191.252:34135 was elected leader, sending a full tablet report...
I20260926 06:53:13.625844 8097 heartbeater.cc:503] Master 127.0.191.252:34135 was elected leader, sending a full tablet report...
I20260926 06:53:13.625964 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.625815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.194:52685)
0926 06:53:13.625915 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.625940 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":64,"threads_started":1}
I20260926 06:53:13.626307 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.625430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.193:50723)
0926 06:53:13.626220 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.626264 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":767,"thread_start_us":70,"threads_started":1}
I20260926 06:53:13.626515 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.624718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.195:58603)
0926 06:53:13.626450 (+ 1732us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.626483 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1665,"thread_start_us":66,"threads_started":1}
I20260926 06:53:13.626832 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.626501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.193:50925)
0926 06:53:13.626607 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.626803 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260926 06:53:13.626861 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.626693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.195:60167)
0926 06:53:13.626804 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.626833 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":74,"threads_started":1}
I20260926 06:53:13.742756 8215 raft_consensus.cc:482] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:53:13.742839 8215 raft_consensus.cc:504] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:13.743223 8215 leader_election.cc:288] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c998e0fe06946d1ac61842d0d139930 (127.0.191.195:41139), 888b433ce9094d4295a686cebda88449 (127.0.191.194:37313)
I20260926 06:53:13.746014 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.743435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:55963 (local address 127.0.191.195:41139)
0926 06:53:13.743491 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:13.743494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:13.743504 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:53:13.743581 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:13.743582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:13.743590 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:13.743658 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:13.743810 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.744282 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.745203 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.745676 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.745768 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:13.745777 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:13.745913 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 06:53:13.745947 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 06:53:13.746014 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.743382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:41139 (local address 127.0.191.193:55963)
0926 06:53:13.743440 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.743452 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:13.743503 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:13.743660 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:13.743661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:13.743664 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:13.743742 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:13.743744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.744357 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.744358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:13.745116 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:13.745118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.745768 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.745769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:13.745842 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:13.745846 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:53:13.745906 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 06:53:13.745947 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 06:53:13.746200 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.743509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:57251 (local address 127.0.191.194:37313)
0926 06:53:13.743546 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:13.743549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:13.743705 (+ 156us) server_negotiation.cc:407] Connection header received
0926 06:53:13.743768 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:13.743769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:13.743779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:13.743843 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:13.743995 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.744451 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.745354 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.745825 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.745902 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:13.745912 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:13.746091 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 06:53:13.746133 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260926 06:53:13.746230 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.743388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:37313 (local address 127.0.191.193:57251)
0926 06:53:13.743628 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.743638 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:13.743710 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:13.743858 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:13.743859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:13.743861 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:13.743924 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:13.743926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.744532 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.744533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:13.745261 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:13.745263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.745932 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.745933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:13.745995 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:13.745998 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:13.746067 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 06:53:13.746154 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":55,"thread_start_us":110,"threads_started":1}
I20260926 06:53:13.746348 8051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fccc665cad4550b78597e4c128b3e4" candidate_uuid: "6a8e80e1b50742878a46ee35b9e33c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c998e0fe06946d1ac61842d0d139930" is_pre_election: true
I20260926 06:53:13.746483 8051 raft_consensus.cc:2454] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8e80e1b50742878a46ee35b9e33c36 in term 0.
I20260926 06:53:13.746485 7915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fccc665cad4550b78597e4c128b3e4" candidate_uuid: "6a8e80e1b50742878a46ee35b9e33c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "888b433ce9094d4295a686cebda88449" is_pre_election: true
I20260926 06:53:13.746553 7915 raft_consensus.cc:2454] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8e80e1b50742878a46ee35b9e33c36 in term 0.
I20260926 06:53:13.746714 7714 leader_election.cc:302] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8e80e1b50742878a46ee35b9e33c36, 888b433ce9094d4295a686cebda88449; no voters:
I20260926 06:53:13.746857 8215 raft_consensus.cc:2781] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:53:13.746908 8215 raft_consensus.cc:482] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:53:13.746929 8215 raft_consensus.cc:3037] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:13.747623 8215 raft_consensus.cc:504] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:13.747761 8215 leader_election.cc:288] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [CANDIDATE]: Term 1 election: Requested vote from peers 2c998e0fe06946d1ac61842d0d139930 (127.0.191.195:41139), 888b433ce9094d4295a686cebda88449 (127.0.191.194:37313)
I20260926 06:53:13.747951 8051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fccc665cad4550b78597e4c128b3e4" candidate_uuid: "6a8e80e1b50742878a46ee35b9e33c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c998e0fe06946d1ac61842d0d139930"
I20260926 06:53:13.747952 7915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fccc665cad4550b78597e4c128b3e4" candidate_uuid: "6a8e80e1b50742878a46ee35b9e33c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "888b433ce9094d4295a686cebda88449"
I20260926 06:53:13.748049 7915 raft_consensus.cc:3037] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:13.748049 8051 raft_consensus.cc:3037] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:13.748804 7915 raft_consensus.cc:2454] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8e80e1b50742878a46ee35b9e33c36 in term 1.
I20260926 06:53:13.748848 8051 raft_consensus.cc:2454] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8e80e1b50742878a46ee35b9e33c36 in term 1.
I20260926 06:53:13.748989 7712 leader_election.cc:302] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c998e0fe06946d1ac61842d0d139930, 6a8e80e1b50742878a46ee35b9e33c36; no voters:
I20260926 06:53:13.749118 8215 raft_consensus.cc:2781] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:13.749274 8215 raft_consensus.cc:686] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [term 1 LEADER]: Becoming Leader. State: Replica: 6a8e80e1b50742878a46ee35b9e33c36, State: Running, Role: LEADER
I20260926 06:53:13.749377 8215 consensus_queue.cc:237] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } }
I20260926 06:53:13.749701 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.749601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.193:34675)
0926 06:53:13.749634 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.749663 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260926 06:53:13.750077 7645 catalog_manager.cc:5495] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a8e80e1b50742878a46ee35b9e33c36 (127.0.191.193). New cstate: current_term: 1 leader_uuid: "6a8e80e1b50742878a46ee35b9e33c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e80e1b50742878a46ee35b9e33c36" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 } health_report { overall_health: UNKNOWN } } }
I20260926 06:53:13.763923 8202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.761725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42486 (local address 127.0.191.193:41465)
0926 06:53:13.761767 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:13.761769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:13.761783 (+ 14us) server_negotiation.cc:407] Connection header received
0926 06:53:13.761813 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:13.761814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:13.761834 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:13.761919 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:53:13.762042 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.762632 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.763324 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:13.763520 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:13.763576 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:13.763669 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:53:13.763763 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:53:13.763816 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:13.763847 (+ 31us) server_negotiation.cc:299] Negotiation successful
0926 06:53:13.763864 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:53:13.764909 7915 raft_consensus.cc:1261] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449 [term 1 FOLLOWER]: Refusing update from remote peer 6a8e80e1b50742878a46ee35b9e33c36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:13.764986 8051 raft_consensus.cc:1261] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930 [term 1 FOLLOWER]: Refusing update from remote peer 6a8e80e1b50742878a46ee35b9e33c36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:13.765213 8220 consensus_queue.cc:1035] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "888b433ce9094d4295a686cebda88449" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:13.765336 8215 consensus_queue.cc:1035] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c998e0fe06946d1ac61842d0d139930" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 41139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:13.766135 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.766014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.194:44735)
0926 06:53:13.766057 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.766091 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260926 06:53:13.766451 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.766334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.195:54603)
0926 06:53:13.766379 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.766407 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 06:53:13.767452 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.767355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.193:40695)
0926 06:53:13.767395 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.767422 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 06:53:13.767684 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.767552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.195:50835)
0926 06:53:13.767623 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.767658 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 06:53:13.768258 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.768158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.194:50881)
0926 06:53:13.768205 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.768228 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 06:53:13.768911 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.768826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37691 (local address 127.0.191.193:37313)
0926 06:53:13.768861 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.768882 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:37691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260926 06:53:13.770107 7643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:13.770201 7643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:13.771332 7643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f40b9326c65f4616a70fbf25924e152e: Sending DeleteTablet for 3 replicas of tablet d6fccc665cad4550b78597e4c128b3e4
I20260926 06:53:13.771644 7758 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fccc665cad4550b78597e4c128b3e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:13 UTC) from {username='oryx', principal='oryx/127.0.191.252@KRBTEST.COM'} at 127.0.0.1:42476
I20260926 06:53:13.771658 8031 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fccc665cad4550b78597e4c128b3e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:13 UTC) from {username='oryx', principal='oryx/127.0.191.252@KRBTEST.COM'} at 127.0.0.1:48546
I20260926 06:53:13.771795 8236 tablet_replica.cc:330] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36: stopping tablet replica
I20260926 06:53:13.771838 8236 raft_consensus.cc:2229] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:53:13.771935 8236 raft_consensus.cc:2258] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:13.772011 7895 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fccc665cad4550b78597e4c128b3e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:13 UTC) from {username='oryx', principal='oryx/127.0.191.252@KRBTEST.COM'} at 127.0.0.1:34984
I20260926 06:53:13.772186 8237 tablet_replica.cc:330] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930: stopping tablet replica
I20260926 06:53:13.772204 8236 ts_tablet_manager.cc:1905] T d6fccc665cad4550b78597e4c128b3e4 P 6a8e80e1b50742878a46ee35b9e33c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:13.772300 8237 raft_consensus.cc:2229] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:13.772368 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 7598
I20260926 06:53:13.773224 8238 tablet_replica.cc:330] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449: stopping tablet replica
I20260926 06:53:13.773284 8238 raft_consensus.cc:2229] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:13.773334 8238 raft_consensus.cc:2258] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:13.773679 8237 raft_consensus.cc:2258] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:13.773711 8238 ts_tablet_manager.cc:1905] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:13.773993 8237 ts_tablet_manager.cc:1905] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:13.775449 8238 ts_tablet_manager.cc:1918] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:13.775524 8238 log.cc:1196] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/d6fccc665cad4550b78597e4c128b3e4
I20260926 06:53:13.775745 8237 ts_tablet_manager.cc:1918] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:13.775779 8238 ts_tablet_manager.cc:1939] T d6fccc665cad4550b78597e4c128b3e4 P 888b433ce9094d4295a686cebda88449: Deleting consensus metadata
I20260926 06:53:13.775811 8237 log.cc:1196] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/d6fccc665cad4550b78597e4c128b3e4
I20260926 06:53:13.776111 7632 catalog_manager.cc:4841] TS 888b433ce9094d4295a686cebda88449 (127.0.191.194:37313): tablet d6fccc665cad4550b78597e4c128b3e4 (table test-table [id=5a4bc9a83e5e4b0182bd5156bd09e9ef]) successfully deleted
I20260926 06:53:13.776149 8237 ts_tablet_manager.cc:1939] T d6fccc665cad4550b78597e4c128b3e4 P 2c998e0fe06946d1ac61842d0d139930: Deleting consensus metadata
I20260926 06:53:13.776444 7630 catalog_manager.cc:4841] TS 2c998e0fe06946d1ac61842d0d139930 (127.0.191.195:41139): tablet d6fccc665cad4550b78597e4c128b3e4 (table test-table [id=5a4bc9a83e5e4b0182bd5156bd09e9ef]) successfully deleted
I20260926 06:53:13.778326 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:13.778194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:41465 (local address 127.0.0.1:42496)
0926 06:53:13.778263 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:13.778291 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:41465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260926 06:53:13.778452 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 7832
W20260926 06:53:13.778462 7632 catalog_manager.cc:4584] TS 6a8e80e1b50742878a46ee35b9e33c36 (127.0.191.193:41465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d6fccc665cad4550b78597e4c128b3e4: Network error: Client connection negotiation failed: client connection to 127.0.191.193:41465: connect: Connection refused (error 111)
I20260926 06:53:13.783989 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 7968
I20260926 06:53:13.789275 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 7317
I20260926 06:53:13.794310 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 7505
2026-09-26T06:53:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7159 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:53:13 dist-test-slave-ws74 krb5kdc[8241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:53:13 dist-test-slave-ws74 krb5kdc[8241](info): set up 2 sockets
Sep 26 06:53:13 dist-test-slave-ws74 krb5kdc[8241](info): commencing operation
krb5kdc: starting...
W20260926 06:53:14.837759 767 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:53:14 dist-test-slave-ws74 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:53:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:14Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.254.keytab.
Entry for principal oryx/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.254.keytab.
I20260926 06:53:14.952432 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:37901
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.254.keytab
--principal=oryx/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:43261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:37901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:15.029938 8254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:15.030169 8254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:15.030205 8254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:15.031349 8254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:15.031421 8254 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 06:53:15.031441 8254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:15.031462 8254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:15.031490 8254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:15.031512 8254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:15.032932 8254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:43261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:37901
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.254.keytab
--principal=oryx/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:37901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:15.033211 8254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:15.033444 8254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:15.038359 8259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:15.038460 8260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:15.038506 8254 server_base.cc:1034] running on GCE node
W20260926 06:53:15.038458 8262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:15.038921 8254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:15.039223 8254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:15.040382 8254 hybrid_clock.cc:648] HybridClock initialized: now 1790405595040370 us; error 26 us; skew 500 ppm
Sep 26 06:53:15 dist-test-slave-ws74 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405595, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:15.042718 8254 init.cc:377] Logged in from keytab as oryx/127.0.191.254@KRBTEST.COM (short username oryx)
I20260926 06:53:15.043113 8254 webserver.cc:460] Webserver started at http://127.0.191.254:38783/ using document root <none> and password file <none>
I20260926 06:53:15.043339 8254 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:15.043402 8254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:15.043541 8254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:15.044521 8254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "c0fbaa6a828a4e168e5c7b7654e27663"
format_stamp: "Formatted at 2026-09-26 06:53:15 on dist-test-slave-ws74"
server_key: "86db832f75e99bbb580de970a8ba75cf"
server_key_iv: "12bfd46123632699e803e7833170992b"
server_key_version: "encryptionkey@0"
I20260926 06:53:15.044860 8254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "c0fbaa6a828a4e168e5c7b7654e27663"
format_stamp: "Formatted at 2026-09-26 06:53:15 on dist-test-slave-ws74"
server_key: "86db832f75e99bbb580de970a8ba75cf"
server_key_iv: "12bfd46123632699e803e7833170992b"
server_key_version: "encryptionkey@0"
I20260926 06:53:15.046161 8254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 06:53:15.046832 8269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:15.046983 8254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:15.047083 8254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "c0fbaa6a828a4e168e5c7b7654e27663"
format_stamp: "Formatted at 2026-09-26 06:53:15 on dist-test-slave-ws74"
server_key: "86db832f75e99bbb580de970a8ba75cf"
server_key_iv: "12bfd46123632699e803e7833170992b"
server_key_version: "encryptionkey@0"
I20260926 06:53:15.047158 8254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:15.081902 8254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:15.083683 8254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:15.083843 8254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:15.088428 8254 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:37901
I20260926 06:53:15.088475 8331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:37901 every 8 connection(s)
I20260926 06:53:15.088806 8254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:15.089468 8332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:15.091642 8332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663: Bootstrap starting.
I20260926 06:53:15.092190 8332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:15.092463 8332 log.cc:824] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:15.093093 8332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663: No bootstrap required, opened a new log
I20260926 06:53:15.095144 8332 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fbaa6a828a4e168e5c7b7654e27663" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37901 } }
I20260926 06:53:15.095302 8332 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:15.095358 8332 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0fbaa6a828a4e168e5c7b7654e27663, State: Initialized, Role: FOLLOWER
I20260926 06:53:15.095530 8332 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fbaa6a828a4e168e5c7b7654e27663" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37901 } }
I20260926 06:53:15.095621 8332 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:15.095664 8332 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:15.095719 8332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:15.096235 8332 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fbaa6a828a4e168e5c7b7654e27663" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37901 } }
I20260926 06:53:15.096345 8332 leader_election.cc:302] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0fbaa6a828a4e168e5c7b7654e27663; no voters:
I20260926 06:53:15.096514 8332 leader_election.cc:288] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:15.096614 8335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:15.096740 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 8254
I20260926 06:53:15.096789 8335 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [term 1 LEADER]: Becoming Leader. State: Replica: c0fbaa6a828a4e168e5c7b7654e27663, State: Running, Role: LEADER
I20260926 06:53:15.096812 8332 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:15.096833 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:15.096874 8335 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fbaa6a828a4e168e5c7b7654e27663" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37901 } }
I20260926 06:53:15.097059 767 external_mini_cluster.cc:1428] Setting key acf1a9055fc3b1917227c35a82905fe5
I20260926 06:53:15.097193 8335 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0fbaa6a828a4e168e5c7b7654e27663. Latest consensus state: current_term: 1 leader_uuid: "c0fbaa6a828a4e168e5c7b7654e27663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fbaa6a828a4e168e5c7b7654e27663" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37901 } } }
I20260926 06:53:15.097256 8335 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:15.097204 8337 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0fbaa6a828a4e168e5c7b7654e27663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fbaa6a828a4e168e5c7b7654e27663" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 37901 } } }
I20260926 06:53:15.097297 8337 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:15.098304 8342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:15.099078 8342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:15.101202 8342 catalog_manager.cc:1347] Generated new cluster ID: 890aca0800fc430881da024c4f34753d
I20260926 06:53:15.101262 8342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:53:15 dist-test-slave-ws74 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
I20260926 06:53:15.104534 8342 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:15.105165 8342 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:15.108433 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.097570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39996 (local address 127.0.191.254:37901)
0926 06:53:15.097763 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.097771 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.097805 (+ 34us) server_negotiation.cc:407] Connection header received
0926 06:53:15.097924 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.097931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.097985 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.098934 (+ 949us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:15.099042 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.100379 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.101151 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.101756 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.101853 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.104658 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:15.104661 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:15.104661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:15.104673 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:15.106636 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.107004 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.107006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.107006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.107094 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.107268 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.107268 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.107268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.107498 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:15.108023 (+ 525us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:15.108088 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 06:53:15.108218 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260926 06:53:15.118040 8342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663: Generated new TSK 0
I20260926 06:53:15.118202 8342 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab.
Entry for principal oryx/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab.
I20260926 06:53:15.142161 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:37901
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab
--principal=oryx/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:43261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:15.221637 8358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:15.221844 8358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:15.221882 8358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:15.223001 8358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:15.223074 8358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:15.223111 8358 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:53:15.223148 8358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:53:15.224576 8358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:43261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab
--principal=oryx/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.191.254:37901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:15.224829 8358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:15.225049 8358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:15.229745 8364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:15.229782 8363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:15.229790 8366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:15.229885 8358 server_base.cc:1034] running on GCE node
I20260926 06:53:15.230140 8358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:15.230360 8358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:15.231508 8358 hybrid_clock.cc:648] HybridClock initialized: now 1790405595231500 us; error 35 us; skew 500 ppm
Sep 26 06:53:15 dist-test-slave-ws74 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405595, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:15.233913 8358 init.cc:377] Logged in from keytab as oryx/127.0.191.193@KRBTEST.COM (short username oryx)
I20260926 06:53:15.234370 8358 webserver.cc:460] Webserver started at http://127.0.191.193:44275/ using document root <none> and password file <none>
I20260926 06:53:15.234597 8358 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:15.234642 8358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:15.234783 8358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:15.235745 8358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "1f79a19aee1343cca2897a371bf4794f"
format_stamp: "Formatted at 2026-09-26 06:53:15 on dist-test-slave-ws74"
server_key: "928b62f03b4e797d09b635d56a80641b"
server_key_iv: "fe905fb8b685da9897dd9e54bbe4c0ae"
server_key_version: "encryptionkey@0"
I20260926 06:53:15.236073 8358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "1f79a19aee1343cca2897a371bf4794f"
format_stamp: "Formatted at 2026-09-26 06:53:15 on dist-test-slave-ws74"
server_key: "928b62f03b4e797d09b635d56a80641b"
server_key_iv: "fe905fb8b685da9897dd9e54bbe4c0ae"
server_key_version: "encryptionkey@0"
I20260926 06:53:15.237339 8358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:53:15.238044 8373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:15.238202 8358 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:15.238286 8358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "1f79a19aee1343cca2897a371bf4794f"
format_stamp: "Formatted at 2026-09-26 06:53:15 on dist-test-slave-ws74"
server_key: "928b62f03b4e797d09b635d56a80641b"
server_key_iv: "fe905fb8b685da9897dd9e54bbe4c0ae"
server_key_version: "encryptionkey@0"
I20260926 06:53:15.238338 8358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:15.250747 8358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:15.252758 8358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:15.252931 8358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:15.253434 8358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:15.253487 8358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:15.253526 8358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:15.253559 8358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:15.260017 8358 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:41675
I20260926 06:53:15.260097 8488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:41675 every 8 connection(s)
Sep 26 06:53:15 dist-test-slave-ws74 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405595, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
I20260926 06:53:15.260351 8358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:53:15.263164 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.253921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:45805 (local address 127.0.191.254:37901)
0926 06:53:15.253982 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.253985 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.254205 (+ 220us) server_negotiation.cc:407] Connection header received
0926 06:53:15.254670 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.254671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.254685 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.254759 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:15.255536 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.256437 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.257983 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.258331 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.258412 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.260860 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:15.260862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:15.260862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:15.260868 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:15.262060 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.262445 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.262445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.262446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.262494 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.262679 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.262680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.262680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.262771 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:15.262817 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:15.263015 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 06:53:15.263105 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:53:15.263331 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.253908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37901 (local address 127.0.191.193:45805)
0926 06:53:15.254064 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:15.254119 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:15.254586 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:15.254820 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:15.254823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:15.255171 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:15.255438 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.255444 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.256529 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.256530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.257886 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.257889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.258413 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.258414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.258519 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.258996 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:15.259005 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:15.260777 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:15.262126 (+ 1349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:15.262131 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:15.262164 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:15.262386 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:15.262544 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:15.262545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:15.262545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:15.262631 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:15.262817 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:15.262821 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:15.262919 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 06:53:15.262993 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 06:53:15.263123 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260926 06:53:15.263796 8489 heartbeater.cc:347] Connected to a master server at 127.0.191.254:37901
I20260926 06:53:15.263873 8489 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:15.264040 8489 heartbeater.cc:511] Master 127.0.191.254:37901 requested a full tablet report, sending...
I20260926 06:53:15.264609 8285 ts_manager.cc:194] Registered new tserver with Master: 1f79a19aee1343cca2897a371bf4794f (127.0.191.193:41675)
I20260926 06:53:15.265527 8285 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:45805
I20260926 06:53:15.266590 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 8358
I20260926 06:53:15.266703 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:53:15.266908 767 external_mini_cluster.cc:1428] Setting key b8a148da11645357239c1fff40aa4e31
I20260926 06:53:15.270572 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.264728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37901 (local address 127.0.191.193:52117)
0926 06:53:15.264864 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:15.264873 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:15.264931 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:15.265040 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:15.265041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:15.265219 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:15.265270 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.265271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.265999 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.266000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.266826 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.266829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.267248 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.267249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.267297 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.267687 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:15.267692 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:15.267946 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:15.269696 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:15.269696 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:15.269697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:15.269894 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:15.270087 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:15.270087 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:15.270088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:15.270134 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:15.270371 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:15.270372 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:15.270404 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 06:53:15.270452 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 06:53:15.270516 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":27,"thread_start_us":89,"threads_started":1}
I20260926 06:53:15.270670 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.264791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:52117 (local address 127.0.191.254:37901)
0926 06:53:15.264827 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.264829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.264921 (+ 92us) server_negotiation.cc:407] Connection header received
0926 06:53:15.264987 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.264987 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.264994 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.265036 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:15.265328 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.265924 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.266898 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.267165 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.267221 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.268041 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:15.268043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:15.268043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:15.268049 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:15.269619 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.269973 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.269974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.269974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.270027 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.270188 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.270188 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.270189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.270315 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:15.270377 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:15.270474 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 06:53:15.270546 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for oryx/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab.
Entry for principal oryx/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab.
I20260926 06:53:15.291502 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:37901
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab
--principal=oryx/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:43261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:15.370005 8498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:15.370194 8498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:15.370218 8498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:15.371397 8498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:15.371436 8498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:15.371479 8498 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:53:15.371512 8498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:53:15.372965 8498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:43261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab
--principal=oryx/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.191.254:37901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:15.373193 8498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:15.373452 8498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:15.378150 8506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:15.378145 8504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:15.378151 8503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:15.378336 8498 server_base.cc:1034] running on GCE node
I20260926 06:53:15.378661 8498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:15.378849 8498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:15.380000 8498 hybrid_clock.cc:648] HybridClock initialized: now 1790405595379986 us; error 26 us; skew 500 ppm
Sep 26 06:53:15 dist-test-slave-ws74 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405595, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:15.382341 8498 init.cc:377] Logged in from keytab as oryx/127.0.191.194@KRBTEST.COM (short username oryx)
I20260926 06:53:15.382711 8498 webserver.cc:460] Webserver started at http://127.0.191.194:44239/ using document root <none> and password file <none>
I20260926 06:53:15.382942 8498 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:15.383004 8498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:15.383147 8498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:15.384087 8498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "125915db2a2e4ec487eed487eb7bdf2f"
format_stamp: "Formatted at 2026-09-26 06:53:15 on dist-test-slave-ws74"
server_key: "f5a2dac564f552140bb2ff29e87fa9c3"
server_key_iv: "5bb8b7b61def75f9c47ee1e82bcd926a"
server_key_version: "encryptionkey@0"
I20260926 06:53:15.384444 8498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "125915db2a2e4ec487eed487eb7bdf2f"
format_stamp: "Formatted at 2026-09-26 06:53:15 on dist-test-slave-ws74"
server_key: "f5a2dac564f552140bb2ff29e87fa9c3"
server_key_iv: "5bb8b7b61def75f9c47ee1e82bcd926a"
server_key_version: "encryptionkey@0"
I20260926 06:53:15.385787 8498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:53:15.386444 8513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:15.386700 8498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:15.386793 8498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "125915db2a2e4ec487eed487eb7bdf2f"
format_stamp: "Formatted at 2026-09-26 06:53:15 on dist-test-slave-ws74"
server_key: "f5a2dac564f552140bb2ff29e87fa9c3"
server_key_iv: "5bb8b7b61def75f9c47ee1e82bcd926a"
server_key_version: "encryptionkey@0"
I20260926 06:53:15.386859 8498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:15.397563 8498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:15.399676 8498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:15.399783 8498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:15.400163 8498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:15.400202 8498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:15.400221 8498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:15.400277 8498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 06:53:15 dist-test-slave-ws74 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405595, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
I20260926 06:53:15.407295 8498 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:38191
I20260926 06:53:15.407351 8628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:38191 every 8 connection(s)
I20260926 06:53:15.407629 8498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:53:15.410223 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.400581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:54705 (local address 127.0.191.254:37901)
0926 06:53:15.400648 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.400650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.400887 (+ 237us) server_negotiation.cc:407] Connection header received
0926 06:53:15.401577 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.401578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.401590 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.401649 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:15.402520 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.403326 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.404990 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.405198 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.405281 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.407692 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:15.407693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:15.407694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:15.407702 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:15.409156 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.409563 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.409564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.409564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.409615 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.409808 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.409808 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.409808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.409894 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:15.409936 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:15.410108 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 06:53:15.410164 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 06:53:15.410418 8522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.400569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37901 (local address 127.0.191.194:54705)
0926 06:53:15.400749 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:15.400796 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:15.401466 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:15.401724 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:15.401728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:15.402127 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:15.402418 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.402424 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.403419 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.403420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.404886 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.404889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.405290 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.405291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.405451 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.405922 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:15.405932 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:15.407604 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:15.409219 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:15.409223 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:15.409225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:15.409487 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:15.409664 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:15.409664 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:15.409665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:15.409766 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:15.409927 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:15.409930 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:15.410024 (+ 94us) client_negotiation.cc:769] Sending connection context
0926 06:53:15.410088 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 06:53:15.410193 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260926 06:53:15.411648 8629 heartbeater.cc:347] Connected to a master server at 127.0.191.254:37901
I20260926 06:53:15.411729 8629 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:15.411900 8629 heartbeater.cc:511] Master 127.0.191.254:37901 requested a full tablet report, sending...
I20260926 06:53:15.412266 8285 ts_manager.cc:194] Registered new tserver with Master: 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194:38191)
I20260926 06:53:15.412663 8285 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:54705
I20260926 06:53:15.415951 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 8498
I20260926 06:53:15.416061 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:53:15.416292 767 external_mini_cluster.cc:1428] Setting key df88f0ef4edf783e2198d503c25583e9
I20260926 06:53:15.417343 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.411898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37901 (local address 127.0.191.194:42429)
0926 06:53:15.412059 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:15.412069 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:15.412129 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:15.412253 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:15.412254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:15.412476 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:15.412532 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.412534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.413208 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.413209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.413786 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.413788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.414139 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.414139 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.414205 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.414603 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:15.414608 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:15.414863 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:15.416344 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:15.416345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:15.416345 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:15.416574 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:15.416734 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:15.416734 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:15.416734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:15.416809 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:15.417129 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:15.417129 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:15.417169 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 06:53:15.417217 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 06:53:15.417288 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260926 06:53:15.417513 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.411979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:42429 (local address 127.0.191.254:37901)
0926 06:53:15.412017 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.412019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.412117 (+ 98us) server_negotiation.cc:407] Connection header received
0926 06:53:15.412177 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.412177 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.412187 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.412248 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:15.412605 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.413124 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.413872 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.414063 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.414136 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.414959 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:15.414960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:15.414960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:15.414966 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:15.416261 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.416636 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.416636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.416636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.416681 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.416872 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.416873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.416873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.417054 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:15.417121 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:15.417257 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 06:53:15.417333 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for oryx/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab.
Entry for principal oryx/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab.
I20260926 06:53:15.439459 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:37901
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab
--principal=oryx/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:43261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:15.518177 8638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:15.518373 8638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:15.518394 8638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:15.519449 8638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:15.519480 8638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:15.519500 8638 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:53:15.519517 8638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:53:15.520864 8638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:43261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab
--principal=oryx/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.191.254:37901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:15.521032 8638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:15.521196 8638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:15.525969 8643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:15.525951 8644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:15.526012 8638 server_base.cc:1034] running on GCE node
W20260926 06:53:15.525969 8646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:15.526362 8638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:15.526588 8638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:15.527745 8638 hybrid_clock.cc:648] HybridClock initialized: now 1790405595527731 us; error 33 us; skew 500 ppm
Sep 26 06:53:15 dist-test-slave-ws74 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405595, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:15.530079 8638 init.cc:377] Logged in from keytab as oryx/127.0.191.195@KRBTEST.COM (short username oryx)
I20260926 06:53:15.530474 8638 webserver.cc:460] Webserver started at http://127.0.191.195:43733/ using document root <none> and password file <none>
I20260926 06:53:15.530691 8638 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:15.530735 8638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:15.530872 8638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:15.531824 8638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "3a84d25399004bd9841ef94e8f8c70fc"
format_stamp: "Formatted at 2026-09-26 06:53:15 on dist-test-slave-ws74"
server_key: "7d8cbd02e7c32f860cc5e76b4693b947"
server_key_iv: "d59cab325383c43b968575f23eb09b4d"
server_key_version: "encryptionkey@0"
I20260926 06:53:15.532150 8638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "3a84d25399004bd9841ef94e8f8c70fc"
format_stamp: "Formatted at 2026-09-26 06:53:15 on dist-test-slave-ws74"
server_key: "7d8cbd02e7c32f860cc5e76b4693b947"
server_key_iv: "d59cab325383c43b968575f23eb09b4d"
server_key_version: "encryptionkey@0"
I20260926 06:53:15.533448 8638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:53:15.534121 8653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:15.534276 8638 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:15.534364 8638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "3a84d25399004bd9841ef94e8f8c70fc"
format_stamp: "Formatted at 2026-09-26 06:53:15 on dist-test-slave-ws74"
server_key: "7d8cbd02e7c32f860cc5e76b4693b947"
server_key_iv: "d59cab325383c43b968575f23eb09b4d"
server_key_version: "encryptionkey@0"
I20260926 06:53:15.534439 8638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:15.570639 8638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:15.572995 8638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:15.573156 8638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:15.573719 8638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:15.573779 8638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:15.573822 8638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:15.573848 8638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 06:53:15 dist-test-slave-ws74 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405595, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
I20260926 06:53:15.580603 8638 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:44011
I20260926 06:53:15.580691 8768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:44011 every 8 connection(s)
I20260926 06:53:15.580979 8638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:53:15.583802 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.574175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:51837 (local address 127.0.191.254:37901)
0926 06:53:15.574251 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.574254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.574441 (+ 187us) server_negotiation.cc:407] Connection header received
0926 06:53:15.574898 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.574900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.574913 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.574993 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:15.575849 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.576671 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.577973 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.578285 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.578381 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.581002 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:15.581003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:15.581004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:15.581011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:15.582466 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.582869 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.582871 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.582871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.582949 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.583190 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.583191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.583192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.583306 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:15.583378 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:15.583628 (+ 250us) server_negotiation.cc:299] Negotiation successful
0926 06:53:15.583705 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 06:53:15.583942 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 8638
I20260926 06:53:15.583972 8662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.574156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37901 (local address 127.0.191.195:51837)
0926 06:53:15.574321 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:15.574360 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:15.574819 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:15.575049 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:15.575070 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:15.575452 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:15.575744 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.575752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.576769 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.576770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.577876 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.577879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.578402 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.578404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.578567 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.579041 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:15.579053 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:15.580913 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:15.582561 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:15.582567 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:15.582569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:15.582784 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:15.583028 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:15.583029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:15.583029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:15.583129 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:15.583369 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:15.583374 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:15.583505 (+ 131us) client_negotiation.cc:769] Sending connection context
0926 06:53:15.583605 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 06:53:15.583750 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":100,"threads_started":1}
I20260926 06:53:15.584057 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:53:15.584265 767 external_mini_cluster.cc:1428] Setting key 57a69728cde905ac26efcd416cb9936d
I20260926 06:53:15.584601 8769 heartbeater.cc:347] Connected to a master server at 127.0.191.254:37901
I20260926 06:53:15.584694 8769 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:15.584875 8769 heartbeater.cc:511] Master 127.0.191.254:37901 requested a full tablet report, sending...
I20260926 06:53:15.585209 8285 ts_manager.cc:194] Registered new tserver with Master: 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195:44011)
I20260926 06:53:15.585616 8285 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:51837
I20260926 06:53:15.590602 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.585834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:46769 (local address 127.0.191.254:37901)
0926 06:53:15.585870 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.585872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.585889 (+ 17us) server_negotiation.cc:407] Connection header received
0926 06:53:15.585909 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.585910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.585919 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.585964 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:15.586283 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.586724 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.587387 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.587582 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.587658 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.588402 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:15.588403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:15.588403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:15.588410 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:15.589543 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.589876 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.589877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.589877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.589927 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.590108 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.590109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.590110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.590226 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:15.590296 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:15.590443 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 06:53:15.590515 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:53:15.590656 8775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.585687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37901 (local address 127.0.191.195:46769)
0926 06:53:15.585807 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:15.585817 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:15.585865 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:15.585977 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:15.585978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:15.586145 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:15.586197 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.586198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.586817 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.586818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.587313 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.587314 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.587659 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.587660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.587720 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.588038 (+ 318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:15.588041 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:15.588323 (+ 282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:15.589623 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:15.589624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:15.589625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:15.589789 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:15.589992 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:15.589993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:15.589993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:15.590047 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:15.590288 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:15.590289 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:15.590333 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 06:53:15.590420 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 06:53:15.590556 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260926 06:53:15.595793 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 06:53:15 dist-test-slave-ws74 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405595, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 06:53:15 dist-test-slave-ws74 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:15 dist-test-slave-ws74 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 06:53:15.616041 8346 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:15.610762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39998 (local address 127.0.191.254:37901)
0926 06:53:15.610821 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.610823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.610871 (+ 48us) server_negotiation.cc:407] Connection header received
0926 06:53:15.610968 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.610970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.610979 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.611043 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:15.611639 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.612172 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.612903 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.613086 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.613182 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.615954 (+ 2772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39998: BlockingRecv error: recv got EOF from 127.0.0.1:39998 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 26 06:53:15 dist-test-slave-ws74 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405595, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
I20260926 06:53:15.623373 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.616860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40008 (local address 127.0.191.254:37901)
0926 06:53:15.616914 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.616916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.617009 (+ 93us) server_negotiation.cc:407] Connection header received
0926 06:53:15.617067 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.617068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.617095 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.617161 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:15.617681 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.618158 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.618913 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.619113 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.619185 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.621101 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:15.621103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:15.621103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:15.621109 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:15.622312 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.622635 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.622636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.622636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.622689 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:15.622888 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:15.622889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:15.622889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:15.622989 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:15.623096 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:15.623199 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 06:53:15.623274 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:53:15.624781 8285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:53:15.626457 8285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:53:15.632469 8791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.628465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:44011 (local address 127.0.0.1:50666)
0926 06:53:15.628724 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:15.628767 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:15.628835 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:15.629630 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:15.629631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:15.629633 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:15.629671 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.629673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.630466 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.630467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.631310 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.631312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.632185 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.632186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.632272 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.632284 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 06:53:15.632353 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 06:53:15.632401 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":1,"thread_start_us":69,"threads_started":1}
I20260926 06:53:15.632519 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.628561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50666 (local address 127.0.191.195:44011)
0926 06:53:15.629499 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.629503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.629513 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:53:15.629548 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.629550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.629564 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.629631 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:15.629725 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.630386 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.631394 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.632000 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.632090 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.632153 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:15.632387 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 06:53:15.632426 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":903,"thread_start_us":88,"threads_started":1}
I20260926 06:53:15.632923 8792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.628650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:41675 (local address 127.0.0.1:59650)
0926 06:53:15.628776 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:15.628786 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:15.628836 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:15.629090 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:15.629093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:15.629094 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:15.629152 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.629154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.630490 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.630491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.631726 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.631728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.632677 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.632678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.632757 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.632760 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:15.632825 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 06:53:15.632864 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":62,"thread_start_us":66,"threads_started":1}
I20260926 06:53:15.633196 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.628764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59650 (local address 127.0.191.193:41675)
0926 06:53:15.628942 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.628947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.628955 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:53:15.628993 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.628995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.629007 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.629087 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:15.629250 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.630400 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.631806 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.632512 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.632575 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.632647 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:15.633064 (+ 417us) server_negotiation.cc:299] Negotiation successful
0926 06:53:15.633104 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":73,"threads_started":1}
I20260926 06:53:15.633213 8703 tablet_service.cc:1467] Processing CreateTablet for tablet 4218dce91a6b4247b42857494c713e16 (DEFAULT_TABLE table=test-table [id=c6be98c31c0842cba2aed0f9e476548f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:15.633590 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.628568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36780 (local address 127.0.191.194:38191)
0926 06:53:15.629203 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.629227 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.629243 (+ 16us) server_negotiation.cc:407] Connection header received
0926 06:53:15.629279 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.629281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.629295 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.629364 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:15.629646 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.630667 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.632073 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.632985 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.633059 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.633127 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:15.633454 (+ 327us) server_negotiation.cc:299] Negotiation successful
0926 06:53:15.633498 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":72,"threads_started":1}
I20260926 06:53:15.633677 8703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4218dce91a6b4247b42857494c713e16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:15.633711 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.628465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:38191 (local address 127.0.0.1:36780)
0926 06:53:15.628758 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:15.628767 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:15.628835 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:15.629469 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:15.629469 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:15.629471 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:15.629514 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.629515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.630759 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.630760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.631977 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.631978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.633158 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.633159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.633216 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.633219 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:15.633290 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 06:53:15.633333 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":2,"thread_start_us":55,"threads_started":1}
I20260926 06:53:15.633852 8423 tablet_service.cc:1467] Processing CreateTablet for tablet 4218dce91a6b4247b42857494c713e16 (DEFAULT_TABLE table=test-table [id=c6be98c31c0842cba2aed0f9e476548f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:15.634164 8423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4218dce91a6b4247b42857494c713e16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:15.634339 8563 tablet_service.cc:1467] Processing CreateTablet for tablet 4218dce91a6b4247b42857494c713e16 (DEFAULT_TABLE table=test-table [id=c6be98c31c0842cba2aed0f9e476548f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:15.634616 8563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4218dce91a6b4247b42857494c713e16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:15.638053 8798 tablet_bootstrap.cc:492] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f: Bootstrap starting.
I20260926 06:53:15.638429 8799 tablet_bootstrap.cc:492] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f: Bootstrap starting.
I20260926 06:53:15.638620 8798 tablet_bootstrap.cc:654] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:15.638625 8797 tablet_bootstrap.cc:492] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc: Bootstrap starting.
I20260926 06:53:15.638972 8798 log.cc:824] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:15.639029 8799 tablet_bootstrap.cc:654] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:15.639155 8797 tablet_bootstrap.cc:654] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:15.639331 8799 log.cc:824] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:15.639483 8797 log.cc:824] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:15.640038 8798 tablet_bootstrap.cc:492] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f: No bootstrap required, opened a new log
I20260926 06:53:15.640107 8798 ts_tablet_manager.cc:1397] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:53:15.640188 8797 tablet_bootstrap.cc:492] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc: No bootstrap required, opened a new log
I20260926 06:53:15.640213 8799 tablet_bootstrap.cc:492] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f: No bootstrap required, opened a new log
I20260926 06:53:15.640259 8797 ts_tablet_manager.cc:1397] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:53:15.640278 8799 ts_tablet_manager.cc:1397] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:53:15.641808 8797 raft_consensus.cc:348] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } }
I20260926 06:53:15.641960 8797 raft_consensus.cc:374] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:15.641996 8797 raft_consensus.cc:729] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a84d25399004bd9841ef94e8f8c70fc, State: Initialized, Role: FOLLOWER
I20260926 06:53:15.642108 8797 consensus_queue.cc:260] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } }
I20260926 06:53:15.642120 8798 raft_consensus.cc:348] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } }
I20260926 06:53:15.642158 8799 raft_consensus.cc:348] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } }
I20260926 06:53:15.642277 8798 raft_consensus.cc:374] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:15.642288 8799 raft_consensus.cc:374] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:15.642328 8798 raft_consensus.cc:729] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f79a19aee1343cca2897a371bf4794f, State: Initialized, Role: FOLLOWER
I20260926 06:53:15.642328 8799 raft_consensus.cc:729] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125915db2a2e4ec487eed487eb7bdf2f, State: Initialized, Role: FOLLOWER
I20260926 06:53:15.642390 8797 ts_tablet_manager.cc:1428] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:53:15.642439 8798 consensus_queue.cc:260] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } }
I20260926 06:53:15.642439 8799 consensus_queue.cc:260] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } }
I20260926 06:53:15.642506 8769 heartbeater.cc:503] Master 127.0.191.254:37901 was elected leader, sending a full tablet report...
I20260926 06:53:15.642696 8798 ts_tablet_manager.cc:1428] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:53:15.642696 8799 ts_tablet_manager.cc:1428] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 06:53:15.642813 8489 heartbeater.cc:503] Master 127.0.191.254:37901 was elected leader, sending a full tablet report...
I20260926 06:53:15.642846 8629 heartbeater.cc:503] Master 127.0.191.254:37901 was elected leader, sending a full tablet report...
W20260926 06:53:15.658164 8630 tablet.cc:2369] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:53:15.761073 8490 tablet.cc:2369] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:53:15.831699 8770 tablet.cc:2369] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:53:15.971424 8804 raft_consensus.cc:482] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:53:15.971565 8804 raft_consensus.cc:504] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } }
I20260926 06:53:15.971984 8804 leader_election.cc:288] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195:44011), 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194:38191)
I20260926 06:53:15.976059 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.972315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:37045 (local address 127.0.191.195:44011)
0926 06:53:15.972356 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.972358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.972418 (+ 60us) server_negotiation.cc:407] Connection header received
0926 06:53:15.972488 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.972489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.972498 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.972556 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:15.972765 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.973544 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.975118 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.975687 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.975766 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.975776 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:15.975921 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 06:53:15.975966 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:53:15.976051 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.972221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:44011 (local address 127.0.191.193:37045)
0926 06:53:15.972339 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:15.972348 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:15.972416 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:15.972579 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:15.972580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:15.972583 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:15.972670 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.972673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.973650 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.973651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.975016 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.975018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.975771 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.975772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.975828 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.975831 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:15.975903 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:53:15.975949 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":88,"thread_start_us":60,"threads_started":1}
I20260926 06:53:15.976123 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.972252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:52905 (local address 127.0.191.194:38191)
0926 06:53:15.972322 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:15.972325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:15.972418 (+ 93us) server_negotiation.cc:407] Connection header received
0926 06:53:15.972487 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:15.972488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:15.972498 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:15.972560 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:15.972765 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.973552 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.975249 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.975803 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.975861 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.975867 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:15.976014 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 06:53:15.976052 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 06:53:15.976163 8806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:15.972178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:38191 (local address 127.0.191.193:52905)
0926 06:53:15.972329 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:15.972341 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:15.972416 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:15.972588 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:15.972589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:15.972591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:15.972693 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.972694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.973651 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.973651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.975015 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:15.975018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:15.975869 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:15.975870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:15.975949 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:15.975951 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:53:15.976002 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 06:53:15.976033 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":77,"thread_start_us":62,"threads_started":1}
I20260926 06:53:15.976537 8723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4218dce91a6b4247b42857494c713e16" candidate_uuid: "1f79a19aee1343cca2897a371bf4794f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a84d25399004bd9841ef94e8f8c70fc" is_pre_election: true
I20260926 06:53:15.976709 8723 raft_consensus.cc:2454] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f79a19aee1343cca2897a371bf4794f in term 0.
I20260926 06:53:15.976724 8583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4218dce91a6b4247b42857494c713e16" candidate_uuid: "1f79a19aee1343cca2897a371bf4794f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125915db2a2e4ec487eed487eb7bdf2f" is_pre_election: true
I20260926 06:53:15.976879 8583 raft_consensus.cc:2454] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f79a19aee1343cca2897a371bf4794f in term 0.
I20260926 06:53:15.976961 8374 leader_election.cc:302] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f79a19aee1343cca2897a371bf4794f, 3a84d25399004bd9841ef94e8f8c70fc; no voters:
I20260926 06:53:15.977156 8804 raft_consensus.cc:2781] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:53:15.977254 8804 raft_consensus.cc:482] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:53:15.977294 8804 raft_consensus.cc:3037] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:15.978004 8804 raft_consensus.cc:504] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } }
I20260926 06:53:15.978152 8804 leader_election.cc:288] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [CANDIDATE]: Term 1 election: Requested vote from peers 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195:44011), 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194:38191)
I20260926 06:53:15.978291 8723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4218dce91a6b4247b42857494c713e16" candidate_uuid: "1f79a19aee1343cca2897a371bf4794f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a84d25399004bd9841ef94e8f8c70fc"
I20260926 06:53:15.978291 8583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4218dce91a6b4247b42857494c713e16" candidate_uuid: "1f79a19aee1343cca2897a371bf4794f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125915db2a2e4ec487eed487eb7bdf2f"
I20260926 06:53:15.978360 8583 raft_consensus.cc:3037] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:15.978374 8723 raft_consensus.cc:3037] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:15.979183 8723 raft_consensus.cc:2454] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f79a19aee1343cca2897a371bf4794f in term 1.
I20260926 06:53:15.979188 8583 raft_consensus.cc:2454] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f79a19aee1343cca2897a371bf4794f in term 1.
I20260926 06:53:15.979333 8374 leader_election.cc:302] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f79a19aee1343cca2897a371bf4794f, 3a84d25399004bd9841ef94e8f8c70fc; no voters:
I20260926 06:53:15.979437 8804 raft_consensus.cc:2781] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:15.979548 8804 raft_consensus.cc:686] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [term 1 LEADER]: Becoming Leader. State: Replica: 1f79a19aee1343cca2897a371bf4794f, State: Running, Role: LEADER
I20260926 06:53:15.979648 8804 consensus_queue.cc:237] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } }
I20260926 06:53:15.980381 8286 catalog_manager.cc:5495] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f79a19aee1343cca2897a371bf4794f (127.0.191.193). New cstate: current_term: 1 leader_uuid: "1f79a19aee1343cca2897a371bf4794f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } health_report { overall_health: HEALTHY } } }
I20260926 06:53:16.070145 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.068081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59666 (local address 127.0.191.193:41675)
0926 06:53:16.068142 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:16.068144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:16.068152 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:53:16.068191 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:16.068192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:16.068208 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:16.068271 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:53:16.068381 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.068844 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.069531 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.069706 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.069769 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.069889 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:53:16.069987 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:53:16.070046 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:16.070074 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 06:53:16.070089 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 06:53:16.071882 8583 raft_consensus.cc:1261] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Refusing update from remote peer 1f79a19aee1343cca2897a371bf4794f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:16.071882 8723 raft_consensus.cc:1261] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Refusing update from remote peer 1f79a19aee1343cca2897a371bf4794f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:16.072192 8804 consensus_queue.cc:1035] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:16.072319 8808 consensus_queue.cc:1035] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [LEADER]: Connected to new peer: Peer: permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:16.074613 8811 mvcc.cc:205] Tried to move back new op lower bound from 7333501321507762176 to 7333501321133297664. Current Snapshot: MvccSnapshot[applied={T|T < 7333501321507762176}]
I20260926 06:53:16.074985 8813 mvcc.cc:205] Tried to move back new op lower bound from 7333501321507762176 to 7333501321133297664. Current Snapshot: MvccSnapshot[applied={T|T < 7333501321507762176}]
I20260926 06:53:16.076766 8286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:16.076860 8286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:16.077541 8286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663: Sending DeleteTablet for 3 replicas of tablet 4218dce91a6b4247b42857494c713e16
I20260926 06:53:16.077896 8563 tablet_service.cc:1514] Processing DeleteTablet for tablet 4218dce91a6b4247b42857494c713e16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:16 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:36780
I20260926 06:53:16.077941 8703 tablet_service.cc:1514] Processing DeleteTablet for tablet 4218dce91a6b4247b42857494c713e16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:16 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:50666
I20260926 06:53:16.078099 8821 tablet_replica.cc:330] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f: stopping tablet replica
I20260926 06:53:16.078110 8423 tablet_service.cc:1514] Processing DeleteTablet for tablet 4218dce91a6b4247b42857494c713e16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:16 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:59650
I20260926 06:53:16.078189 8821 raft_consensus.cc:2229] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:16.078169 8286 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 06:53:16.078289 8821 raft_consensus.cc:2258] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 06:53:16.078305 8286 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:53:16.078722 8821 ts_tablet_manager.cc:1905] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:16.078737 8822 tablet_replica.cc:330] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc: stopping tablet replica
I20260926 06:53:16.078824 8822 raft_consensus.cc:2229] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:16.078986 8822 raft_consensus.cc:2258] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:16.079707 8703 tablet_service.cc:1467] Processing CreateTablet for tablet 425936b00204452695e020997a1a8bbd (DEFAULT_TABLE table=test-table [id=ced30680feb846fca40f808e6c3cb225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:16.079813 8823 tablet_replica.cc:330] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f: stopping tablet replica
I20260926 06:53:16.079829 8703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 425936b00204452695e020997a1a8bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:16.079892 8823 raft_consensus.cc:2229] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:53:16.079998 8823 raft_consensus.cc:2258] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:16.080369 8823 ts_tablet_manager.cc:1905] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:16.080482 8821 ts_tablet_manager.cc:1918] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:16.080542 8821 log.cc:1196] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/4218dce91a6b4247b42857494c713e16
I20260926 06:53:16.080787 8821 ts_tablet_manager.cc:1939] T 4218dce91a6b4247b42857494c713e16 P 125915db2a2e4ec487eed487eb7bdf2f: Deleting consensus metadata
I20260926 06:53:16.081214 8797 tablet_bootstrap.cc:492] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc: Bootstrap starting.
I20260926 06:53:16.081283 8423 tablet_service.cc:1467] Processing CreateTablet for tablet 425936b00204452695e020997a1a8bbd (DEFAULT_TABLE table=test-table [id=ced30680feb846fca40f808e6c3cb225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:16.081400 8423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 425936b00204452695e020997a1a8bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:16.081651 8797 tablet_bootstrap.cc:654] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:16.081969 8823 ts_tablet_manager.cc:1918] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:16.082083 8823 log.cc:1196] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/wal/wals/4218dce91a6b4247b42857494c713e16
I20260926 06:53:16.082170 8797 tablet_bootstrap.cc:492] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc: No bootstrap required, opened a new log
I20260926 06:53:16.082207 8797 ts_tablet_manager.cc:1397] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 06:53:16.082324 8823 ts_tablet_manager.cc:1939] T 4218dce91a6b4247b42857494c713e16 P 1f79a19aee1343cca2897a371bf4794f: Deleting consensus metadata
I20260926 06:53:16.082345 8797 raft_consensus.cc:348] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.082423 8797 raft_consensus.cc:374] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:16.082444 8797 raft_consensus.cc:729] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a84d25399004bd9841ef94e8f8c70fc, State: Initialized, Role: FOLLOWER
I20260926 06:53:16.082472 8797 consensus_queue.cc:260] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.082551 8797 ts_tablet_manager.cc:1428] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:16.082760 8798 tablet_bootstrap.cc:492] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f: Bootstrap starting.
I20260926 06:53:16.083092 8798 tablet_bootstrap.cc:654] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:16.083525 8272 catalog_manager.cc:4841] TS 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194:38191): tablet 4218dce91a6b4247b42857494c713e16 (table test-table [id=c6be98c31c0842cba2aed0f9e476548f]) successfully deleted
I20260926 06:53:16.083606 8272 catalog_manager.cc:4841] TS 1f79a19aee1343cca2897a371bf4794f (127.0.191.193:41675): tablet 4218dce91a6b4247b42857494c713e16 (table test-table [id=c6be98c31c0842cba2aed0f9e476548f]) successfully deleted
I20260926 06:53:16.083715 8563 tablet_service.cc:1467] Processing CreateTablet for tablet 425936b00204452695e020997a1a8bbd (DEFAULT_TABLE table=test-table [id=ced30680feb846fca40f808e6c3cb225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:16.083818 8563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 425936b00204452695e020997a1a8bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:16.084712 8799 tablet_bootstrap.cc:492] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f: Bootstrap starting.
I20260926 06:53:16.085107 8799 tablet_bootstrap.cc:654] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:16.085394 8798 tablet_bootstrap.cc:492] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f: No bootstrap required, opened a new log
I20260926 06:53:16.085444 8798 ts_tablet_manager.cc:1397] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260926 06:53:16.085580 8798 raft_consensus.cc:348] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.085659 8798 raft_consensus.cc:374] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:16.085693 8798 raft_consensus.cc:729] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f79a19aee1343cca2897a371bf4794f, State: Initialized, Role: FOLLOWER
I20260926 06:53:16.085742 8798 consensus_queue.cc:260] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.085801 8799 tablet_bootstrap.cc:492] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f: No bootstrap required, opened a new log
I20260926 06:53:16.085839 8798 ts_tablet_manager.cc:1428] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:16.085845 8799 ts_tablet_manager.cc:1397] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 06:53:16.085989 8799 raft_consensus.cc:348] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.086063 8799 raft_consensus.cc:374] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:16.086086 8799 raft_consensus.cc:729] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125915db2a2e4ec487eed487eb7bdf2f, State: Initialized, Role: FOLLOWER
I20260926 06:53:16.086126 8799 consensus_queue.cc:260] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.086156 8822 ts_tablet_manager.cc:1905] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:16.086220 8799 ts_tablet_manager.cc:1428] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 06:53:16.087574 8822 ts_tablet_manager.cc:1918] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:16.087627 8822 log.cc:1196] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/4218dce91a6b4247b42857494c713e16
I20260926 06:53:16.087845 8822 ts_tablet_manager.cc:1939] T 4218dce91a6b4247b42857494c713e16 P 3a84d25399004bd9841ef94e8f8c70fc: Deleting consensus metadata
I20260926 06:53:16.088196 8270 catalog_manager.cc:4841] TS 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195:44011): tablet 4218dce91a6b4247b42857494c713e16 (table test-table [id=c6be98c31c0842cba2aed0f9e476548f]) successfully deleted
I20260926 06:53:16.197290 8816 raft_consensus.cc:482] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:53:16.197407 8816 raft_consensus.cc:504] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.197576 8816 leader_election.cc:288] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194:38191), 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195:44011)
I20260926 06:53:16.197821 8583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "425936b00204452695e020997a1a8bbd" candidate_uuid: "1f79a19aee1343cca2897a371bf4794f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125915db2a2e4ec487eed487eb7bdf2f" is_pre_election: true
I20260926 06:53:16.197821 8723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "425936b00204452695e020997a1a8bbd" candidate_uuid: "1f79a19aee1343cca2897a371bf4794f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a84d25399004bd9841ef94e8f8c70fc" is_pre_election: true
I20260926 06:53:16.197929 8723 raft_consensus.cc:2454] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f79a19aee1343cca2897a371bf4794f in term 0.
I20260926 06:53:16.197929 8583 raft_consensus.cc:2454] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f79a19aee1343cca2897a371bf4794f in term 0.
I20260926 06:53:16.198102 8376 leader_election.cc:302] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125915db2a2e4ec487eed487eb7bdf2f, 1f79a19aee1343cca2897a371bf4794f; no voters:
I20260926 06:53:16.198215 8816 raft_consensus.cc:2781] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:53:16.198247 8816 raft_consensus.cc:482] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:53:16.198277 8816 raft_consensus.cc:3037] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:16.199105 8816 raft_consensus.cc:504] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.199245 8816 leader_election.cc:288] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [CANDIDATE]: Term 1 election: Requested vote from peers 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194:38191), 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195:44011)
I20260926 06:53:16.199410 8583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "425936b00204452695e020997a1a8bbd" candidate_uuid: "1f79a19aee1343cca2897a371bf4794f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125915db2a2e4ec487eed487eb7bdf2f"
I20260926 06:53:16.199441 8723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "425936b00204452695e020997a1a8bbd" candidate_uuid: "1f79a19aee1343cca2897a371bf4794f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a84d25399004bd9841ef94e8f8c70fc"
I20260926 06:53:16.199487 8583 raft_consensus.cc:3037] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:16.199499 8723 raft_consensus.cc:3037] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:16.200354 8723 raft_consensus.cc:2454] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f79a19aee1343cca2897a371bf4794f in term 1.
I20260926 06:53:16.200531 8374 leader_election.cc:302] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f79a19aee1343cca2897a371bf4794f, 3a84d25399004bd9841ef94e8f8c70fc; no voters:
I20260926 06:53:16.200615 8583 raft_consensus.cc:2454] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f79a19aee1343cca2897a371bf4794f in term 1.
I20260926 06:53:16.200634 8816 raft_consensus.cc:2781] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:16.200678 8816 raft_consensus.cc:686] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [term 1 LEADER]: Becoming Leader. State: Replica: 1f79a19aee1343cca2897a371bf4794f, State: Running, Role: LEADER
I20260926 06:53:16.200722 8816 consensus_queue.cc:237] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.201280 8286 catalog_manager.cc:5495] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f79a19aee1343cca2897a371bf4794f (127.0.191.193). New cstate: current_term: 1 leader_uuid: "1f79a19aee1343cca2897a371bf4794f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } health_report { overall_health: UNKNOWN } } }
Sep 26 06:53:16 dist-test-slave-ws74 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405595, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.254@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
I20260926 06:53:16.260483 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.254040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:37901 (local address 127.0.0.1:40012)
0926 06:53:16.254187 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:16.254199 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:16.254275 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:16.254405 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:16.254406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:16.254696 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:16.254753 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.254755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.255374 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.255375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.255893 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.255894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.256207 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.256208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.256248 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.256699 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:16.256703 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:16.258059 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:16.259447 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:16.259453 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:16.259454 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:16.259640 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:16.259806 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:16.259806 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:16.259807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:16.259905 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:16.260156 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:16.260162 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:16.260211 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 06:53:16.260294 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 06:53:16.260389 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260926 06:53:16.260766 8834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.254106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40012 (local address 127.0.191.254:37901)
0926 06:53:16.254267 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:16.254268 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:16.254277 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:16.254322 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:16.254323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:16.254331 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:16.254389 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:16.254816 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.255314 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.255957 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.256150 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.256198 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.258132 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:16.258133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:16.258133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:16.258139 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:16.259379 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:16.259700 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:16.259700 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:16.259701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:16.259753 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:16.259972 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:16.259973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:16.259973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:16.260094 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:16.260147 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:16.260462 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 06:53:16.260671 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260926 06:53:16.262797 8286 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:40012:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 06:53:16.262974 8286 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:53:16.264247 8423 tablet_service.cc:1467] Processing CreateTablet for tablet ee06025bb5484238a181f0c08bdbcc5b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0333912c53694d76ab7ebb64e8f187d5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:53:16.264323 8563 tablet_service.cc:1467] Processing CreateTablet for tablet ee06025bb5484238a181f0c08bdbcc5b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0333912c53694d76ab7ebb64e8f187d5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:53:16.264402 8423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee06025bb5484238a181f0c08bdbcc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:16.264406 8703 tablet_service.cc:1467] Processing CreateTablet for tablet ee06025bb5484238a181f0c08bdbcc5b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0333912c53694d76ab7ebb64e8f187d5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:53:16.264472 8563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee06025bb5484238a181f0c08bdbcc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:16.264523 8703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee06025bb5484238a181f0c08bdbcc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:16.265767 8799 tablet_bootstrap.cc:492] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f: Bootstrap starting.
I20260926 06:53:16.265926 8798 tablet_bootstrap.cc:492] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f: Bootstrap starting.
I20260926 06:53:16.265995 8797 tablet_bootstrap.cc:492] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc: Bootstrap starting.
I20260926 06:53:16.266184 8798 tablet_bootstrap.cc:654] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:16.266184 8799 tablet_bootstrap.cc:654] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:16.266350 8797 tablet_bootstrap.cc:654] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:16.266812 8798 tablet_bootstrap.cc:492] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f: No bootstrap required, opened a new log
I20260926 06:53:16.266868 8798 ts_tablet_manager.cc:1397] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:16.266973 8797 tablet_bootstrap.cc:492] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc: No bootstrap required, opened a new log
I20260926 06:53:16.267014 8797 ts_tablet_manager.cc:1397] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 06:53:16.266974 8798 raft_consensus.cc:348] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.267061 8798 raft_consensus.cc:374] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:16.267089 8798 raft_consensus.cc:729] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f79a19aee1343cca2897a371bf4794f, State: Initialized, Role: FOLLOWER
I20260926 06:53:16.267143 8798 consensus_queue.cc:260] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.267174 8797 raft_consensus.cc:348] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.267234 8798 ts_tablet_manager.cc:1428] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:16.267249 8797 raft_consensus.cc:374] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:16.267280 8797 raft_consensus.cc:729] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a84d25399004bd9841ef94e8f8c70fc, State: Initialized, Role: FOLLOWER
I20260926 06:53:16.267252 8816 tablet_replica.cc:439] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } }
I20260926 06:53:16.267333 8816 tablet_replica.cc:442] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:53:16.267326 8797 consensus_queue.cc:260] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.267414 8797 ts_tablet_manager.cc:1428] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 06:53:16.267446 8799 tablet_bootstrap.cc:492] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f: No bootstrap required, opened a new log
I20260926 06:53:16.267498 8799 ts_tablet_manager.cc:1397] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:53:16.267560 8803 tablet_replica.cc:439] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } }
I20260926 06:53:16.267645 8803 tablet_replica.cc:442] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:53:16.267635 8799 raft_consensus.cc:348] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.267700 8799 raft_consensus.cc:374] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:16.267727 8799 raft_consensus.cc:729] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125915db2a2e4ec487eed487eb7bdf2f, State: Initialized, Role: FOLLOWER
I20260926 06:53:16.267779 8799 consensus_queue.cc:260] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.267854 8799 ts_tablet_manager.cc:1428] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 06:53:16.267985 8805 tablet_replica.cc:439] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } }
I20260926 06:53:16.268072 8805 tablet_replica.cc:442] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:53:16.287217 8803 raft_consensus.cc:482] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:53:16.287290 8803 raft_consensus.cc:504] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.287588 8803 leader_election.cc:288] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f79a19aee1343cca2897a371bf4794f (127.0.191.193:41675), 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194:38191)
I20260926 06:53:16.290683 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.288029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:36611 (local address 127.0.191.194:38191)
0926 06:53:16.288071 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:16.288073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:16.288156 (+ 83us) server_negotiation.cc:407] Connection header received
0926 06:53:16.288225 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:16.288226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:16.288236 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:16.288290 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:16.288425 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.288897 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.289962 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.290359 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.290443 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.290451 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:16.290585 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 06:53:16.290617 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:53:16.290697 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.287982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:38191 (local address 127.0.191.195:36611)
0926 06:53:16.288086 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:16.288096 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:16.288170 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:16.288294 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:16.288295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:16.288296 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:16.288361 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.288362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.288972 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.288973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.289882 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.289884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.290444 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.290445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.290506 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.290509 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:16.290576 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 06:53:16.290615 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":46,"threads_started":1}
I20260926 06:53:16.290782 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.287833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:39269 (local address 127.0.191.193:41675)
0926 06:53:16.287886 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:16.287888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:16.288009 (+ 121us) server_negotiation.cc:407] Connection header received
0926 06:53:16.288103 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:16.288104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:16.288114 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:16.288141 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:16.288311 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.288975 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.290092 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.290446 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.290533 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.290541 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:16.290687 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 06:53:16.290723 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 06:53:16.290978 8838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.287776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:41675 (local address 127.0.191.195:39269)
0926 06:53:16.287947 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:16.287958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:16.288034 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:16.288199 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:16.288200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:16.288201 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:16.288248 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.288250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.289051 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.289051 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.290014 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.290017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.290534 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.290535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.290600 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.290604 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:53:16.290679 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:53:16.290718 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":82,"threads_started":1}
I20260926 06:53:16.291157 8583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee06025bb5484238a181f0c08bdbcc5b" candidate_uuid: "3a84d25399004bd9841ef94e8f8c70fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125915db2a2e4ec487eed487eb7bdf2f" is_pre_election: true
I20260926 06:53:16.291256 8583 raft_consensus.cc:2454] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a84d25399004bd9841ef94e8f8c70fc in term 0.
I20260926 06:53:16.291193 8443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee06025bb5484238a181f0c08bdbcc5b" candidate_uuid: "3a84d25399004bd9841ef94e8f8c70fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f79a19aee1343cca2897a371bf4794f" is_pre_election: true
I20260926 06:53:16.291317 8443 raft_consensus.cc:2454] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a84d25399004bd9841ef94e8f8c70fc in term 0.
I20260926 06:53:16.291437 8656 leader_election.cc:302] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125915db2a2e4ec487eed487eb7bdf2f, 3a84d25399004bd9841ef94e8f8c70fc; no voters:
I20260926 06:53:16.291563 8803 raft_consensus.cc:2781] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:53:16.291602 8803 raft_consensus.cc:482] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:53:16.291632 8803 raft_consensus.cc:3037] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:16.292255 8803 raft_consensus.cc:504] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.292371 8803 leader_election.cc:288] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [CANDIDATE]: Term 1 election: Requested vote from peers 1f79a19aee1343cca2897a371bf4794f (127.0.191.193:41675), 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194:38191)
I20260926 06:53:16.292493 8443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee06025bb5484238a181f0c08bdbcc5b" candidate_uuid: "3a84d25399004bd9841ef94e8f8c70fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f79a19aee1343cca2897a371bf4794f"
I20260926 06:53:16.292514 8583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee06025bb5484238a181f0c08bdbcc5b" candidate_uuid: "3a84d25399004bd9841ef94e8f8c70fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125915db2a2e4ec487eed487eb7bdf2f"
I20260926 06:53:16.292567 8443 raft_consensus.cc:3037] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:16.292574 8583 raft_consensus.cc:3037] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:16.293232 8583 raft_consensus.cc:2454] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a84d25399004bd9841ef94e8f8c70fc in term 1.
I20260926 06:53:16.293339 8443 raft_consensus.cc:2454] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a84d25399004bd9841ef94e8f8c70fc in term 1.
I20260926 06:53:16.293427 8656 leader_election.cc:302] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125915db2a2e4ec487eed487eb7bdf2f, 3a84d25399004bd9841ef94e8f8c70fc; no voters:
I20260926 06:53:16.293515 8803 raft_consensus.cc:2781] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:16.293617 8803 raft_consensus.cc:686] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [term 1 LEADER]: Becoming Leader. State: Replica: 3a84d25399004bd9841ef94e8f8c70fc, State: Running, Role: LEADER
I20260926 06:53:16.293682 8803 consensus_queue.cc:237] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.294005 8840 tablet_replica.cc:439] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc: TxnStatusTablet state changed. Reason: New leader 3a84d25399004bd9841ef94e8f8c70fc. Latest consensus state: current_term: 1 leader_uuid: "3a84d25399004bd9841ef94e8f8c70fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } }
I20260926 06:53:16.294100 8840 tablet_replica.cc:442] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:53:16.294268 8842 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 06:53:16.294371 8842 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 06:53:16.294302 8285 catalog_manager.cc:5495] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195). New cstate: current_term: 1 leader_uuid: "3a84d25399004bd9841ef94e8f8c70fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } health_report { overall_health: HEALTHY } } }
I20260926 06:53:16.305505 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.302820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50678 (local address 127.0.191.195:44011)
0926 06:53:16.302866 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:16.302868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:16.302942 (+ 74us) server_negotiation.cc:407] Connection header received
0926 06:53:16.303048 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:16.303049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:16.303071 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:16.303126 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:53:16.303261 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.303707 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.304776 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.304945 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.304992 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.305181 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:53:16.305273 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:53:16.305324 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:16.305427 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 06:53:16.305450 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:53:16.305546 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.302752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:44011 (local address 127.0.0.1:50678)
0926 06:53:16.302890 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:16.302902 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:16.302963 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:16.303146 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:16.303147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:16.303148 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:53:16.303198 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.303202 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.303782 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.303783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.304697 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.304699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.305013 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.305014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.305081 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.305096 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:53:16.305332 (+ 236us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:53:16.305339 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 06:53:16.305418 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 06:53:16.305466 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260926 06:53:16.306577 8583 raft_consensus.cc:1261] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Refusing update from remote peer 3a84d25399004bd9841ef94e8f8c70fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:16.306600 8443 raft_consensus.cc:1261] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Refusing update from remote peer 3a84d25399004bd9841ef94e8f8c70fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:16.306855 8803 consensus_queue.cc:1035] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:16.306972 8840 consensus_queue.cc:1035] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:16.307969 8805 tablet_replica.cc:439] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f: TxnStatusTablet state changed. Reason: New leader 3a84d25399004bd9841ef94e8f8c70fc. Latest consensus state: current_term: 1 leader_uuid: "3a84d25399004bd9841ef94e8f8c70fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } }
I20260926 06:53:16.308058 8805 tablet_replica.cc:442] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:53:16.308112 8816 tablet_replica.cc:439] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f: TxnStatusTablet state changed. Reason: New leader 3a84d25399004bd9841ef94e8f8c70fc. Latest consensus state: current_term: 1 leader_uuid: "3a84d25399004bd9841ef94e8f8c70fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } }
I20260926 06:53:16.308185 8816 tablet_replica.cc:442] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:53:16.308468 8840 tablet_replica.cc:439] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a84d25399004bd9841ef94e8f8c70fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } }
I20260926 06:53:16.308538 8840 tablet_replica.cc:442] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:53:16.308638 8840 tablet_replica.cc:439] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a84d25399004bd9841ef94e8f8c70fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } }
I20260926 06:53:16.308673 8840 tablet_replica.cc:442] T ee06025bb5484238a181f0c08bdbcc5b P 3a84d25399004bd9841ef94e8f8c70fc: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:53:16.308722 8805 tablet_replica.cc:439] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a84d25399004bd9841ef94e8f8c70fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } }
I20260926 06:53:16.308777 8812 mvcc.cc:205] Tried to move back new op lower bound from 7333501322469781504 to 7333501322419445760. Current Snapshot: MvccSnapshot[applied={T|T < 7333501322469781504 or (T in {7333501322469781504})}]
I20260926 06:53:16.308815 8805 tablet_replica.cc:442] T ee06025bb5484238a181f0c08bdbcc5b P 125915db2a2e4ec487eed487eb7bdf2f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:53:16.309477 8816 tablet_replica.cc:439] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a84d25399004bd9841ef94e8f8c70fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } } }
I20260926 06:53:16.309556 8816 tablet_replica.cc:442] T ee06025bb5484238a181f0c08bdbcc5b P 1f79a19aee1343cca2897a371bf4794f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:53:16.315016 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.312531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:36503 (local address 127.0.191.195:44011)
0926 06:53:16.312577 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:16.312579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:16.312640 (+ 61us) server_negotiation.cc:407] Connection header received
0926 06:53:16.312730 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:16.312731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:16.312742 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:16.312780 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:53:16.312901 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.313351 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.314472 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.314637 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.314685 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.314852 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:53:16.314874 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:53:16.314910 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:16.314960 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 06:53:16.314974 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 06:53:16.315090 8850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.312456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:44011 (local address 127.0.191.193:36503)
0926 06:53:16.312586 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:16.312597 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:16.312656 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:16.312792 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:16.312793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:16.312794 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:53:16.312850 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.312852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.313468 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.313469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.314392 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.314395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.314702 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.314703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.314768 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.314782 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:53:16.314911 (+ 129us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:53:16.314912 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:53:16.314959 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 06:53:16.315008 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260926 06:53:16.316854 8723 raft_consensus.cc:1261] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Refusing update from remote peer 1f79a19aee1343cca2897a371bf4794f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:16.316857 8583 raft_consensus.cc:1261] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Refusing update from remote peer 1f79a19aee1343cca2897a371bf4794f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:16.317045 8804 consensus_queue.cc:1035] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:16.317124 8816 consensus_queue.cc:1035] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [LEADER]: Connected to new peer: Peer: permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:16.327771 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.322138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:35075 (local address 127.0.191.193:41675)
0926 06:53:16.322190 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:16.322192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:16.325571 (+ 3379us) server_negotiation.cc:407] Connection header received
0926 06:53:16.325630 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:16.325631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:16.325641 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:16.325695 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:53:16.325852 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.326369 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.327111 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.327302 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.327369 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.327540 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:53:16.327575 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:53:16.327631 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:16.327689 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 06:53:16.327707 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:53:16.327877 8855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.321960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:41675 (local address 127.0.191.195:35075)
0926 06:53:16.325516 (+ 3556us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:16.325527 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:16.325583 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:16.325697 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:16.325698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:16.325699 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:53:16.325758 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.325759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.326441 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.326441 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.327021 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.327022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.327390 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.327391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.327444 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.327460 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:53:16.327631 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:53:16.327633 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:53:16.327684 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 06:53:16.327719 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3508,"thread_start_us":78,"threads_started":1}
I20260926 06:53:16.336501 8286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:16.336596 8286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:16.337146 8286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663: Sending DeleteTablet for 3 replicas of tablet 425936b00204452695e020997a1a8bbd
I20260926 06:53:16.337360 8423 tablet_service.cc:1514] Processing DeleteTablet for tablet 425936b00204452695e020997a1a8bbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:16 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:59650
I20260926 06:53:16.337432 8563 tablet_service.cc:1514] Processing DeleteTablet for tablet 425936b00204452695e020997a1a8bbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:16 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:36780
I20260926 06:53:16.337468 8823 tablet_replica.cc:330] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f: stopping tablet replica
I20260926 06:53:16.337519 8823 raft_consensus.cc:2229] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:53:16.337518 8821 tablet_replica.cc:330] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f: stopping tablet replica
I20260926 06:53:16.337567 8821 raft_consensus.cc:2229] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:16.337576 8823 raft_consensus.cc:2258] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:16.337600 8821 raft_consensus.cc:2258] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:16.337857 8823 ts_tablet_manager.cc:1905] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:16.337857 8821 ts_tablet_manager.cc:1905] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:16.337873 8703 tablet_service.cc:1514] Processing DeleteTablet for tablet 425936b00204452695e020997a1a8bbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:16 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:50666
I20260926 06:53:16.337998 8822 tablet_replica.cc:330] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc: stopping tablet replica
I20260926 06:53:16.338045 8822 raft_consensus.cc:2229] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:16.338083 8822 raft_consensus.cc:2258] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:16.339859 8823 ts_tablet_manager.cc:1918] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:53:16.339922 8823 log.cc:1196] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/wal/wals/425936b00204452695e020997a1a8bbd
I20260926 06:53:16.339944 8822 ts_tablet_manager.cc:1905] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:16.340143 8823 ts_tablet_manager.cc:1939] T 425936b00204452695e020997a1a8bbd P 1f79a19aee1343cca2897a371bf4794f: Deleting consensus metadata
I20260926 06:53:16.340433 8272 catalog_manager.cc:4841] TS 1f79a19aee1343cca2897a371bf4794f (127.0.191.193:41675): tablet 425936b00204452695e020997a1a8bbd (table test-table [id=ced30680feb846fca40f808e6c3cb225]) successfully deleted
I20260926 06:53:16.341485 8822 ts_tablet_manager.cc:1918] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:53:16.341543 8822 log.cc:1196] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/425936b00204452695e020997a1a8bbd
I20260926 06:53:16.341763 8822 ts_tablet_manager.cc:1939] T 425936b00204452695e020997a1a8bbd P 3a84d25399004bd9841ef94e8f8c70fc: Deleting consensus metadata
I20260926 06:53:16.342044 8821 ts_tablet_manager.cc:1918] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:53:16.342095 8821 log.cc:1196] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/425936b00204452695e020997a1a8bbd
I20260926 06:53:16.342073 8270 catalog_manager.cc:4841] TS 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195:44011): tablet 425936b00204452695e020997a1a8bbd (table test-table [id=ced30680feb846fca40f808e6c3cb225]) successfully deleted
I20260926 06:53:16.342293 8821 ts_tablet_manager.cc:1939] T 425936b00204452695e020997a1a8bbd P 125915db2a2e4ec487eed487eb7bdf2f: Deleting consensus metadata
I20260926 06:53:16.342533 8272 catalog_manager.cc:4841] TS 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194:38191): tablet 425936b00204452695e020997a1a8bbd (table test-table [id=ced30680feb846fca40f808e6c3cb225]) successfully deleted
Sep 26 06:53:16 dist-test-slave-ws74 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 06:53:16 dist-test-slave-ws74 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:16 dist-test-slave-ws74 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 06:53:16.361548 8834 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:16.355622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.0.191.254:37901)
0926 06:53:16.355672 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:16.355675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:16.355819 (+ 144us) server_negotiation.cc:407] Connection header received
0926 06:53:16.355890 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:16.355891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:16.355901 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:16.355947 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:16.356614 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.357089 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.358236 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.358433 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.358478 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.361476 (+ 2998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40016: BlockingRecv error: recv got EOF from 127.0.0.1:40016 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 26 06:53:16 dist-test-slave-ws74 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM
I20260926 06:53:16.369238 8834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.362289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40020 (local address 127.0.191.254:37901)
0926 06:53:16.362326 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:16.362327 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:16.362386 (+ 59us) server_negotiation.cc:407] Connection header received
0926 06:53:16.362463 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:16.362463 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:16.362470 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:16.362512 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:16.363047 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.363525 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.364459 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.364629 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.364678 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.366914 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:16.366915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:16.366916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:16.366923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:16.368178 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:16.368505 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:16.368506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:16.368506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:16.368559 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:16.368760 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:16.368760 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:16.368761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:16.368861 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:16.368962 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:16.369061 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 06:53:16.369131 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:53:16.370643 8286 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:53:16.370807 8286 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:53:16.372011 8423 tablet_service.cc:1467] Processing CreateTablet for tablet 5009bcb13bb646e5a6d9bf33bb65ce04 (DEFAULT_TABLE table=test-table [id=503128f96309400d8ac33e5bcf4d123a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:16.372046 8563 tablet_service.cc:1467] Processing CreateTablet for tablet 5009bcb13bb646e5a6d9bf33bb65ce04 (DEFAULT_TABLE table=test-table [id=503128f96309400d8ac33e5bcf4d123a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:16.372145 8423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5009bcb13bb646e5a6d9bf33bb65ce04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:16.372200 8563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5009bcb13bb646e5a6d9bf33bb65ce04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:16.373101 8799 tablet_bootstrap.cc:492] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f: Bootstrap starting.
I20260926 06:53:16.373154 8703 tablet_service.cc:1467] Processing CreateTablet for tablet 5009bcb13bb646e5a6d9bf33bb65ce04 (DEFAULT_TABLE table=test-table [id=503128f96309400d8ac33e5bcf4d123a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:16.373265 8703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5009bcb13bb646e5a6d9bf33bb65ce04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:16.373488 8798 tablet_bootstrap.cc:492] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f: Bootstrap starting.
I20260926 06:53:16.373567 8799 tablet_bootstrap.cc:654] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:16.373831 8798 tablet_bootstrap.cc:654] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:16.374397 8798 tablet_bootstrap.cc:492] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f: No bootstrap required, opened a new log
I20260926 06:53:16.374451 8798 ts_tablet_manager.cc:1397] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 06:53:16.374562 8798 raft_consensus.cc:348] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.374629 8798 raft_consensus.cc:374] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:16.374650 8798 raft_consensus.cc:729] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f79a19aee1343cca2897a371bf4794f, State: Initialized, Role: FOLLOWER
I20260926 06:53:16.374696 8798 consensus_queue.cc:260] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.374703 8797 tablet_bootstrap.cc:492] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc: Bootstrap starting.
I20260926 06:53:16.374806 8798 ts_tablet_manager.cc:1428] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:16.374979 8797 tablet_bootstrap.cc:654] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:16.375495 8797 tablet_bootstrap.cc:492] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc: No bootstrap required, opened a new log
I20260926 06:53:16.375542 8797 ts_tablet_manager.cc:1397] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 06:53:16.375638 8797 raft_consensus.cc:348] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.375694 8797 raft_consensus.cc:374] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:16.375698 8799 tablet_bootstrap.cc:492] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f: No bootstrap required, opened a new log
I20260926 06:53:16.375713 8797 raft_consensus.cc:729] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a84d25399004bd9841ef94e8f8c70fc, State: Initialized, Role: FOLLOWER
I20260926 06:53:16.375736 8799 ts_tablet_manager.cc:1397] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 06:53:16.375772 8797 consensus_queue.cc:260] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.375852 8797 ts_tablet_manager.cc:1428] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:16.375888 8799 raft_consensus.cc:348] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.375958 8799 raft_consensus.cc:374] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:16.375973 8799 raft_consensus.cc:729] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125915db2a2e4ec487eed487eb7bdf2f, State: Initialized, Role: FOLLOWER
I20260926 06:53:16.376021 8799 consensus_queue.cc:260] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.376106 8799 ts_tablet_manager.cc:1428] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:16.427590 8805 raft_consensus.cc:482] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:53:16.427693 8805 raft_consensus.cc:504] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.428071 8805 leader_election.cc:288] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f79a19aee1343cca2897a371bf4794f (127.0.191.193:41675), 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195:44011)
I20260926 06:53:16.431845 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.428441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:44295 (local address 127.0.191.195:44011)
0926 06:53:16.428484 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:16.428486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:16.428495 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:16.428523 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:16.428524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:16.428532 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:16.428590 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:16.428796 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.429359 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.430841 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.431379 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.431425 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.431433 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:16.431728 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 06:53:16.431775 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:53:16.431895 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.428250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:41675 (local address 127.0.191.194:45029)
0926 06:53:16.428518 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:16.428527 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:16.428574 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:16.428758 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:16.428759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:16.428760 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:16.428819 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.428821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.429720 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.429721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.430825 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.430827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.431555 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.431556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.431632 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.431634 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:53:16.431714 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 06:53:16.431777 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
I20260926 06:53:16.432032 8872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.428249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:44011 (local address 127.0.191.194:44295)
0926 06:53:16.428387 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:16.428398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:16.428456 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:16.428642 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:16.428642 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:16.428645 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:16.428700 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.428702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.429501 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.429502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.430738 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:16.430740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.431577 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.431578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:16.431631 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.431634 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:16.431716 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 06:53:16.431777 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":49,"thread_start_us":96,"threads_started":1}
I20260926 06:53:16.432068 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.428288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:45029 (local address 127.0.191.193:41675)
0926 06:53:16.428338 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:16.428340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:16.428570 (+ 230us) server_negotiation.cc:407] Connection header received
0926 06:53:16.428676 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:16.428677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:16.428691 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:16.428755 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:16.428873 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.429613 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.430913 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.431443 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.431483 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.431490 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:16.431967 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 06:53:16.432007 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:53:16.432381 8723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5009bcb13bb646e5a6d9bf33bb65ce04" candidate_uuid: "125915db2a2e4ec487eed487eb7bdf2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a84d25399004bd9841ef94e8f8c70fc" is_pre_election: true
I20260926 06:53:16.432472 8723 raft_consensus.cc:2454] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 125915db2a2e4ec487eed487eb7bdf2f in term 0.
I20260926 06:53:16.432492 8443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5009bcb13bb646e5a6d9bf33bb65ce04" candidate_uuid: "125915db2a2e4ec487eed487eb7bdf2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f79a19aee1343cca2897a371bf4794f" is_pre_election: true
I20260926 06:53:16.432586 8443 raft_consensus.cc:2454] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 125915db2a2e4ec487eed487eb7bdf2f in term 0.
I20260926 06:53:16.432622 8514 leader_election.cc:302] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125915db2a2e4ec487eed487eb7bdf2f, 3a84d25399004bd9841ef94e8f8c70fc; no voters:
I20260926 06:53:16.432721 8805 raft_consensus.cc:2781] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:53:16.432757 8805 raft_consensus.cc:482] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:53:16.432776 8805 raft_consensus.cc:3037] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:16.433650 8805 raft_consensus.cc:504] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.433794 8805 leader_election.cc:288] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [CANDIDATE]: Term 1 election: Requested vote from peers 1f79a19aee1343cca2897a371bf4794f (127.0.191.193:41675), 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195:44011)
I20260926 06:53:16.433926 8443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5009bcb13bb646e5a6d9bf33bb65ce04" candidate_uuid: "125915db2a2e4ec487eed487eb7bdf2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f79a19aee1343cca2897a371bf4794f"
I20260926 06:53:16.433936 8723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5009bcb13bb646e5a6d9bf33bb65ce04" candidate_uuid: "125915db2a2e4ec487eed487eb7bdf2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a84d25399004bd9841ef94e8f8c70fc"
I20260926 06:53:16.433998 8443 raft_consensus.cc:3037] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:16.434005 8723 raft_consensus.cc:3037] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:16.434578 8723 raft_consensus.cc:2454] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125915db2a2e4ec487eed487eb7bdf2f in term 1.
I20260926 06:53:16.434717 8443 raft_consensus.cc:2454] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125915db2a2e4ec487eed487eb7bdf2f in term 1.
I20260926 06:53:16.434723 8514 leader_election.cc:302] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125915db2a2e4ec487eed487eb7bdf2f, 3a84d25399004bd9841ef94e8f8c70fc; no voters:
I20260926 06:53:16.434880 8805 raft_consensus.cc:2781] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:16.435022 8805 raft_consensus.cc:686] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [term 1 LEADER]: Becoming Leader. State: Replica: 125915db2a2e4ec487eed487eb7bdf2f, State: Running, Role: LEADER
I20260926 06:53:16.435086 8805 consensus_queue.cc:237] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.435721 8285 catalog_manager.cc:5495] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194). New cstate: current_term: 1 leader_uuid: "125915db2a2e4ec487eed487eb7bdf2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } health_report { overall_health: UNKNOWN } } }
I20260926 06:53:16.450322 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.448000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36794 (local address 127.0.191.194:38191)
0926 06:53:16.448071 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:16.448073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:16.448082 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:16.448169 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:16.448169 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:16.448188 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:16.448277 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:53:16.448388 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.448839 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.449598 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.449793 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.449860 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.450013 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:53:16.450110 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:53:16.450178 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:16.450218 (+ 40us) server_negotiation.cc:299] Negotiation successful
0926 06:53:16.450244 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 06:53:16.451122 8443 raft_consensus.cc:1261] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Refusing update from remote peer 125915db2a2e4ec487eed487eb7bdf2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:16.451119 8723 raft_consensus.cc:1261] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Refusing update from remote peer 125915db2a2e4ec487eed487eb7bdf2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:16.451380 8874 consensus_queue.cc:1035] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:16.451535 8805 consensus_queue.cc:1035] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:16.454670 8286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:16.454767 8286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:16.455361 8286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663: Sending DeleteTablet for 3 replicas of tablet 5009bcb13bb646e5a6d9bf33bb65ce04
I20260926 06:53:16.455595 8423 tablet_service.cc:1514] Processing DeleteTablet for tablet 5009bcb13bb646e5a6d9bf33bb65ce04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:16 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:59650
I20260926 06:53:16.455688 8703 tablet_service.cc:1514] Processing DeleteTablet for tablet 5009bcb13bb646e5a6d9bf33bb65ce04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:16 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:50666
I20260926 06:53:16.455780 8823 tablet_replica.cc:330] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f: stopping tablet replica
I20260926 06:53:16.455839 8823 raft_consensus.cc:2229] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:16.455807 8286 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 06:53:16.455888 8823 raft_consensus.cc:2258] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 06:53:16.455942 8286 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:53:16.455955 8822 tablet_replica.cc:330] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc: stopping tablet replica
I20260926 06:53:16.455999 8822 raft_consensus.cc:2229] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:16.456040 8822 raft_consensus.cc:2258] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:16.456398 8563 tablet_service.cc:1514] Processing DeleteTablet for tablet 5009bcb13bb646e5a6d9bf33bb65ce04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:16 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:36780
I20260926 06:53:16.456481 8821 tablet_replica.cc:330] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f: stopping tablet replica
I20260926 06:53:16.456485 8822 ts_tablet_manager.cc:1905] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:16.456532 8821 raft_consensus.cc:2229] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:53:16.456627 8821 raft_consensus.cc:2258] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:16.457633 8823 ts_tablet_manager.cc:1905] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:16.457664 8821 ts_tablet_manager.cc:1905] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:16.457911 8822 ts_tablet_manager.cc:1918] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:16.457960 8822 log.cc:1196] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/5009bcb13bb646e5a6d9bf33bb65ce04
I20260926 06:53:16.458159 8822 ts_tablet_manager.cc:1939] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 3a84d25399004bd9841ef94e8f8c70fc: Deleting consensus metadata
I20260926 06:53:16.458453 8270 catalog_manager.cc:4841] TS 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195:44011): tablet 5009bcb13bb646e5a6d9bf33bb65ce04 (table test-table [id=503128f96309400d8ac33e5bcf4d123a]) successfully deleted
I20260926 06:53:16.459075 8823 ts_tablet_manager.cc:1918] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:16.459133 8823 log.cc:1196] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-0/wal/wals/5009bcb13bb646e5a6d9bf33bb65ce04
I20260926 06:53:16.459105 8821 ts_tablet_manager.cc:1918] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:16.459172 8821 log.cc:1196] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/5009bcb13bb646e5a6d9bf33bb65ce04
I20260926 06:53:16.459359 8823 ts_tablet_manager.cc:1939] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 1f79a19aee1343cca2897a371bf4794f: Deleting consensus metadata
I20260926 06:53:16.459375 8821 ts_tablet_manager.cc:1939] T 5009bcb13bb646e5a6d9bf33bb65ce04 P 125915db2a2e4ec487eed487eb7bdf2f: Deleting consensus metadata
I20260926 06:53:16.459723 8272 catalog_manager.cc:4841] TS 1f79a19aee1343cca2897a371bf4794f (127.0.191.193:41675): tablet 5009bcb13bb646e5a6d9bf33bb65ce04 (table test-table [id=503128f96309400d8ac33e5bcf4d123a]) successfully deleted
I20260926 06:53:16.459805 8272 catalog_manager.cc:4841] TS 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194:38191): tablet 5009bcb13bb646e5a6d9bf33bb65ce04 (table test-table [id=503128f96309400d8ac33e5bcf4d123a]) successfully deleted
I20260926 06:53:16.460494 8563 tablet_service.cc:1467] Processing CreateTablet for tablet 4de6c5027c23438e925379f5c7fb12af (DEFAULT_TABLE table=test-table [id=a8c9b012872a4019bfedf37c7a439cab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:16.460494 8423 tablet_service.cc:1467] Processing CreateTablet for tablet 4de6c5027c23438e925379f5c7fb12af (DEFAULT_TABLE table=test-table [id=a8c9b012872a4019bfedf37c7a439cab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:16.460593 8563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4de6c5027c23438e925379f5c7fb12af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:16.460594 8423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4de6c5027c23438e925379f5c7fb12af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:16.461516 8703 tablet_service.cc:1467] Processing CreateTablet for tablet 4de6c5027c23438e925379f5c7fb12af (DEFAULT_TABLE table=test-table [id=a8c9b012872a4019bfedf37c7a439cab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:16.461615 8703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4de6c5027c23438e925379f5c7fb12af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:16.461774 8798 tablet_bootstrap.cc:492] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f: Bootstrap starting.
I20260926 06:53:16.461774 8799 tablet_bootstrap.cc:492] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f: Bootstrap starting.
I20260926 06:53:16.462100 8798 tablet_bootstrap.cc:654] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:16.462168 8799 tablet_bootstrap.cc:654] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:16.462765 8799 tablet_bootstrap.cc:492] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f: No bootstrap required, opened a new log
I20260926 06:53:16.462796 8797 tablet_bootstrap.cc:492] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc: Bootstrap starting.
I20260926 06:53:16.462809 8799 ts_tablet_manager.cc:1397] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:16.462870 8798 tablet_bootstrap.cc:492] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f: No bootstrap required, opened a new log
I20260926 06:53:16.462911 8798 ts_tablet_manager.cc:1397] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 06:53:16.462951 8799 raft_consensus.cc:348] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.463013 8799 raft_consensus.cc:374] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:16.463049 8799 raft_consensus.cc:729] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125915db2a2e4ec487eed487eb7bdf2f, State: Initialized, Role: FOLLOWER
I20260926 06:53:16.463018 8798 raft_consensus.cc:348] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.463092 8798 raft_consensus.cc:374] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:16.463100 8797 tablet_bootstrap.cc:654] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:16.463114 8798 raft_consensus.cc:729] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f79a19aee1343cca2897a371bf4794f, State: Initialized, Role: FOLLOWER
I20260926 06:53:16.463091 8799 consensus_queue.cc:260] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.463155 8798 consensus_queue.cc:260] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.463213 8799 ts_tablet_manager.cc:1428] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:16.463236 8798 ts_tablet_manager.cc:1428] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:16.463752 8797 tablet_bootstrap.cc:492] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc: No bootstrap required, opened a new log
I20260926 06:53:16.463805 8797 ts_tablet_manager.cc:1397] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 06:53:16.463917 8797 raft_consensus.cc:348] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.463999 8797 raft_consensus.cc:374] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:16.464021 8797 raft_consensus.cc:729] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a84d25399004bd9841ef94e8f8c70fc, State: Initialized, Role: FOLLOWER
I20260926 06:53:16.464090 8797 consensus_queue.cc:260] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.464184 8797 ts_tablet_manager.cc:1428] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:16.690666 8808 raft_consensus.cc:482] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:53:16.690781 8808 raft_consensus.cc:504] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.690929 8808 leader_election.cc:288] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194:38191), 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195:44011)
I20260926 06:53:16.691109 8583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4de6c5027c23438e925379f5c7fb12af" candidate_uuid: "1f79a19aee1343cca2897a371bf4794f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125915db2a2e4ec487eed487eb7bdf2f" is_pre_election: true
I20260926 06:53:16.691146 8723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4de6c5027c23438e925379f5c7fb12af" candidate_uuid: "1f79a19aee1343cca2897a371bf4794f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a84d25399004bd9841ef94e8f8c70fc" is_pre_election: true
I20260926 06:53:16.691221 8583 raft_consensus.cc:2454] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f79a19aee1343cca2897a371bf4794f in term 0.
I20260926 06:53:16.691221 8723 raft_consensus.cc:2454] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f79a19aee1343cca2897a371bf4794f in term 0.
I20260926 06:53:16.691371 8376 leader_election.cc:302] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125915db2a2e4ec487eed487eb7bdf2f, 1f79a19aee1343cca2897a371bf4794f; no voters:
I20260926 06:53:16.691494 8808 raft_consensus.cc:2781] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:53:16.691541 8808 raft_consensus.cc:482] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:53:16.691555 8808 raft_consensus.cc:3037] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:16.692198 8808 raft_consensus.cc:504] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.692351 8808 leader_election.cc:288] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [CANDIDATE]: Term 1 election: Requested vote from peers 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194:38191), 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195:44011)
I20260926 06:53:16.692479 8583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4de6c5027c23438e925379f5c7fb12af" candidate_uuid: "1f79a19aee1343cca2897a371bf4794f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125915db2a2e4ec487eed487eb7bdf2f"
I20260926 06:53:16.692552 8583 raft_consensus.cc:3037] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:16.692516 8723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4de6c5027c23438e925379f5c7fb12af" candidate_uuid: "1f79a19aee1343cca2897a371bf4794f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a84d25399004bd9841ef94e8f8c70fc"
I20260926 06:53:16.692593 8723 raft_consensus.cc:3037] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:16.693115 8583 raft_consensus.cc:2454] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f79a19aee1343cca2897a371bf4794f in term 1.
I20260926 06:53:16.693176 8723 raft_consensus.cc:2454] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f79a19aee1343cca2897a371bf4794f in term 1.
I20260926 06:53:16.693329 8376 leader_election.cc:302] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125915db2a2e4ec487eed487eb7bdf2f, 1f79a19aee1343cca2897a371bf4794f; no voters:
I20260926 06:53:16.693456 8808 raft_consensus.cc:2781] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:16.693514 8808 raft_consensus.cc:686] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [term 1 LEADER]: Becoming Leader. State: Replica: 1f79a19aee1343cca2897a371bf4794f, State: Running, Role: LEADER
I20260926 06:53:16.693545 8808 consensus_queue.cc:237] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } }
I20260926 06:53:16.694156 8284 catalog_manager.cc:5495] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f79a19aee1343cca2897a371bf4794f (127.0.191.193). New cstate: current_term: 1 leader_uuid: "1f79a19aee1343cca2897a371bf4794f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79a19aee1343cca2897a371bf4794f" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 41675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 } health_report { overall_health: UNKNOWN } } }
I20260926 06:53:16.748239 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.746406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59670 (local address 127.0.191.193:41675)
0926 06:53:16.746459 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:16.746461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:16.746469 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:53:16.746506 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:16.746506 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:16.746514 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:16.746555 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:53:16.746633 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.747088 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.747717 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:16.747891 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:16.747939 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:16.748038 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:53:16.748061 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:53:16.748093 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:16.748130 (+ 37us) server_negotiation.cc:299] Negotiation successful
0926 06:53:16.748145 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:53:16.750180 8723 raft_consensus.cc:1261] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Refusing update from remote peer 1f79a19aee1343cca2897a371bf4794f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:16.750173 8583 raft_consensus.cc:1261] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Refusing update from remote peer 1f79a19aee1343cca2897a371bf4794f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:16.750392 8808 consensus_queue.cc:1035] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [LEADER]: Connected to new peer: Peer: permanent_uuid: "125915db2a2e4ec487eed487eb7bdf2f" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 38191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:16.750545 8804 consensus_queue.cc:1035] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a84d25399004bd9841ef94e8f8c70fc" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 44011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:16.760177 8286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:16.760260 8286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:16.760757 8286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0fbaa6a828a4e168e5c7b7654e27663: Sending DeleteTablet for 3 replicas of tablet 4de6c5027c23438e925379f5c7fb12af
I20260926 06:53:16.760977 8703 tablet_service.cc:1514] Processing DeleteTablet for tablet 4de6c5027c23438e925379f5c7fb12af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:16 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:50666
I20260926 06:53:16.761020 8563 tablet_service.cc:1514] Processing DeleteTablet for tablet 4de6c5027c23438e925379f5c7fb12af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:16 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:36780
I20260926 06:53:16.761075 8822 tablet_replica.cc:330] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc: stopping tablet replica
I20260926 06:53:16.761063 8423 tablet_service.cc:1514] Processing DeleteTablet for tablet 4de6c5027c23438e925379f5c7fb12af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:16 UTC) from {username='oryx', principal='oryx/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:59650
I20260926 06:53:16.761114 8821 tablet_replica.cc:330] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f: stopping tablet replica
I20260926 06:53:16.761132 8822 raft_consensus.cc:2229] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:16.761142 8823 tablet_replica.cc:330] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f: stopping tablet replica
I20260926 06:53:16.761161 8822 raft_consensus.cc:2258] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:16.761159 8821 raft_consensus.cc:2229] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:53:16.761193 8821 raft_consensus.cc:2258] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:16.761188 8823 raft_consensus.cc:2229] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:53:16.761251 8823 raft_consensus.cc:2258] T 4de6c5027c23438e925379f5c7fb12af P 1f79a19aee1343cca2897a371bf4794f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:16.761508 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 8358
I20260926 06:53:16.761524 8822 ts_tablet_manager.cc:1905] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:16.761588 8821 ts_tablet_manager.cc:1905] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:16.763172 8822 ts_tablet_manager.cc:1918] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:53:16.763172 8821 ts_tablet_manager.cc:1918] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:53:16.763228 8821 log.cc:1196] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/4de6c5027c23438e925379f5c7fb12af
I20260926 06:53:16.763228 8822 log.cc:1196] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/4de6c5027c23438e925379f5c7fb12af
I20260926 06:53:16.763460 8821 ts_tablet_manager.cc:1939] T 4de6c5027c23438e925379f5c7fb12af P 125915db2a2e4ec487eed487eb7bdf2f: Deleting consensus metadata
I20260926 06:53:16.763469 8822 ts_tablet_manager.cc:1939] T 4de6c5027c23438e925379f5c7fb12af P 3a84d25399004bd9841ef94e8f8c70fc: Deleting consensus metadata
I20260926 06:53:16.763727 8272 catalog_manager.cc:4841] TS 125915db2a2e4ec487eed487eb7bdf2f (127.0.191.194:38191): tablet 4de6c5027c23438e925379f5c7fb12af (table test-table [id=a8c9b012872a4019bfedf37c7a439cab]) successfully deleted
I20260926 06:53:16.764792 8270 catalog_manager.cc:4841] TS 3a84d25399004bd9841ef94e8f8c70fc (127.0.191.195:44011): tablet 4de6c5027c23438e925379f5c7fb12af (table test-table [id=a8c9b012872a4019bfedf37c7a439cab]) successfully deleted
W20260926 06:53:16.768018 8272 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:41675 (error 108)
I20260926 06:53:16.768358 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 8498
I20260926 06:53:16.769093 8888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:16.768400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:41675 (local address 127.0.0.1:59672)
0926 06:53:16.769002 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:16.769048 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:41675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":69,"threads_started":1}
W20260926 06:53:16.769181 8272 catalog_manager.cc:4584] TS 1f79a19aee1343cca2897a371bf4794f (127.0.191.193:41675): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4de6c5027c23438e925379f5c7fb12af: Network error: Client connection negotiation failed: client connection to 127.0.191.193:41675: connect: Connection refused (error 111)
I20260926 06:53:16.774415 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 8638
I20260926 06:53:16.780056 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 8254
2026-09-26T06:53:16Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2983 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:53:16 dist-test-slave-ws74 krb5kdc[8891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:53:16 dist-test-slave-ws74 krb5kdc[8891](info): set up 2 sockets
Sep 26 06:53:16 dist-test-slave-ws74 krb5kdc[8891](info): commencing operation
krb5kdc: starting...
W20260926 06:53:17.813858 767 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:53:17 dist-test-slave-ws74 krb5kdc[8891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:53:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:53:17.923914 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:43651
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:43651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:18.000944 8904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:18.001175 8904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:18.001211 8904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:18.002406 8904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:18.002470 8904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:18.002496 8904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:18.002528 8904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:18.002554 8904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:18.004153 8904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:43651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:43651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:18.004415 8904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:18.004657 8904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:53:18.009343 8904 server_base.cc:1034] running on GCE node
W20260926 06:53:18.009307 8910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:18.009305 8912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:18.009320 8909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:18.009778 8904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:18.010030 8904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:18.011162 8904 hybrid_clock.cc:648] HybridClock initialized: now 1790405598011144 us; error 35 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T06:53:18Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1240 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:53:18 dist-test-slave-ws74 krb5kdc[8917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:53:18 dist-test-slave-ws74 krb5kdc[8917](info): set up 2 sockets
Sep 26 06:53:18 dist-test-slave-ws74 krb5kdc[8917](info): commencing operation
krb5kdc: starting...
W20260926 06:53:19.066214 767 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:53:19 dist-test-slave-ws74 krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:53:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:53:19.179883 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:35967
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:35967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:19.258890 8930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:19.259110 8930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:19.259145 8930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:19.260262 8930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:19.260320 8930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:19.260345 8930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:19.260366 8930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:19.260392 8930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:19.261845 8930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:35967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:35967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:19.262100 8930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:19.262302 8930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:19.266918 8935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:19.266939 8936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:19.266940 8938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:19.267001 8930 server_base.cc:1034] running on GCE node
I20260926 06:53:19.267339 8930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:19.267598 8930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:19.268761 8930 hybrid_clock.cc:648] HybridClock initialized: now 1790405599268743 us; error 34 us; skew 500 ppm
Sep 26 06:53:19 dist-test-slave-ws74 krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:19.276525 8930 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:53:19.276968 8930 webserver.cc:460] Webserver started at http://127.0.191.254:37969/ using document root <none> and password file <none>
I20260926 06:53:19.277190 8930 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:19.277262 8930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:19.277446 8930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:19.278482 8930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "17d3cd84a01942e6b70283f101026ea8"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "1c21089c4b6a717d4d22772192d6866e"
server_key_iv: "1c6b0cdd606de55b15c17f538d748620"
server_key_version: "encryptionkey@0"
I20260926 06:53:19.278831 8930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "17d3cd84a01942e6b70283f101026ea8"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "1c21089c4b6a717d4d22772192d6866e"
server_key_iv: "1c6b0cdd606de55b15c17f538d748620"
server_key_version: "encryptionkey@0"
I20260926 06:53:19.280077 8930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:53:19.280759 8945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.280918 8930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:19.281009 8930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "17d3cd84a01942e6b70283f101026ea8"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "1c21089c4b6a717d4d22772192d6866e"
server_key_iv: "1c6b0cdd606de55b15c17f538d748620"
server_key_version: "encryptionkey@0"
I20260926 06:53:19.281086 8930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:19.292160 8930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:19.294478 8930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:19.294637 8930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:19.298732 8930 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:35967
I20260926 06:53:19.298748 8997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:35967 every 8 connection(s)
I20260926 06:53:19.299280 8930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:19.300096 8998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:19.302551 8998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8: Bootstrap starting.
I20260926 06:53:19.303233 8998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:19.303486 8998 log.cc:824] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:19.303817 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 8930
I20260926 06:53:19.303937 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:19.304162 767 external_mini_cluster.cc:1428] Setting key 360b22b661405b5767085d0bb8fcac44
I20260926 06:53:19.304349 8998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8: No bootstrap required, opened a new log
I20260926 06:53:19.307046 8998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d3cd84a01942e6b70283f101026ea8" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 35967 } }
I20260926 06:53:19.307195 8998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:19.307220 8998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17d3cd84a01942e6b70283f101026ea8, State: Initialized, Role: FOLLOWER
I20260926 06:53:19.307371 8998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d3cd84a01942e6b70283f101026ea8" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 35967 } }
I20260926 06:53:19.307443 8998 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:19.307484 8998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:19.307536 8998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:19.308120 8998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d3cd84a01942e6b70283f101026ea8" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 35967 } }
I20260926 06:53:19.308205 8998 leader_election.cc:302] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17d3cd84a01942e6b70283f101026ea8; no voters:
I20260926 06:53:19.308446 8998 leader_election.cc:288] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:19.308468 9003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:19.308626 9003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 1 LEADER]: Becoming Leader. State: Replica: 17d3cd84a01942e6b70283f101026ea8, State: Running, Role: LEADER
I20260926 06:53:19.308749 8998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:19.308786 9003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d3cd84a01942e6b70283f101026ea8" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 35967 } }
I20260926 06:53:19.309172 9003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17d3cd84a01942e6b70283f101026ea8. Latest consensus state: current_term: 1 leader_uuid: "17d3cd84a01942e6b70283f101026ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d3cd84a01942e6b70283f101026ea8" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 35967 } } }
I20260926 06:53:19.309185 9005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17d3cd84a01942e6b70283f101026ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d3cd84a01942e6b70283f101026ea8" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 35967 } } }
I20260926 06:53:19.309321 9003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:19.309625 9005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:19.309847 9011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:19.310645 9011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 06:53:19 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:19.312726 9011 catalog_manager.cc:1347] Generated new cluster ID: 6e1705e77d88457b8b8aad7f0efc7335
I20260926 06:53:19.313041 9011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:53:19.315661 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:19.304662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37742 (local address 127.0.191.254:35967)
0926 06:53:19.304852 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:19.304862 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:19.304897 (+ 35us) server_negotiation.cc:407] Connection header received
0926 06:53:19.305657 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:19.305664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:19.305713 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:19.305858 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:19.306500 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.307472 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.308274 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.308870 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.310296 (+ 1426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:19.311932 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:19.311936 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:19.311936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:19.311945 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:19.313903 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:19.314360 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:19.314363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:19.314363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:19.314456 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:19.314693 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:19.314694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:19.314694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:19.314957 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:19.315099 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:19.315247 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 06:53:19.315465 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260926 06:53:19.327819 9011 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:19.328599 9011 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:19.345965 9011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8: Generated new TSK 0
I20260926 06:53:19.346185 9011 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:53:19.372098 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:35967
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:19.450882 9024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:19.451094 9024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:19.451133 9024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:19.452251 9024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:19.452310 9024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:19.452354 9024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:53:19.453893 9024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:35967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:19.454142 9024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:19.454351 9024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:19.459070 9030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:19.459075 9029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:19.459131 9032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:19.459120 9024 server_base.cc:1034] running on GCE node
I20260926 06:53:19.459419 9024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:19.459650 9024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:19.460784 9024 hybrid_clock.cc:648] HybridClock initialized: now 1790405599460766 us; error 36 us; skew 500 ppm
Sep 26 06:53:19 dist-test-slave-ws74 krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:19.463231 9024 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:53:19.463696 9024 webserver.cc:460] Webserver started at http://127.0.191.193:44145/ using document root <none> and password file <none>
I20260926 06:53:19.463879 9024 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:19.463918 9024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:19.464000 9024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:19.464908 9024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "ab0257bb3f6146ee8ddff40dd4f7bcc0"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "fdb3de5932b4f447002529e0c7893b4c"
server_key_iv: "2f4bf18d268d36d5007ed799eca831ee"
server_key_version: "encryptionkey@0"
I20260926 06:53:19.465189 9024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "ab0257bb3f6146ee8ddff40dd4f7bcc0"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "fdb3de5932b4f447002529e0c7893b4c"
server_key_iv: "2f4bf18d268d36d5007ed799eca831ee"
server_key_version: "encryptionkey@0"
I20260926 06:53:19.466442 9024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:53:19.467094 9039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.467264 9024 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.467358 9024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "ab0257bb3f6146ee8ddff40dd4f7bcc0"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "fdb3de5932b4f447002529e0c7893b4c"
server_key_iv: "2f4bf18d268d36d5007ed799eca831ee"
server_key_version: "encryptionkey@0"
I20260926 06:53:19.467427 9024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:19.481825 9024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:19.484058 9024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:19.484203 9024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:19.484431 9024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:19.484791 9024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:19.484833 9024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.484870 9024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:19.484900 9024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.490545 9024 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:45307
I20260926 06:53:19.490597 9152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:45307 every 8 connection(s)
I20260926 06:53:19.490917 9024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:53:19 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:19.496749 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 9024
I20260926 06:53:19.496863 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:53:19.497056 767 external_mini_cluster.cc:1428] Setting key d799f473189ede6d2a0f03caeda31166
I20260926 06:53:19.499747 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:19.491586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:59157 (local address 127.0.191.254:35967)
0926 06:53:19.491641 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:19.491643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:19.491827 (+ 184us) server_negotiation.cc:407] Connection header received
0926 06:53:19.492710 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:19.492711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:19.492719 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:19.492780 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:19.493319 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.493871 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.494786 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.494968 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.495016 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:19.496879 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:19.496881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:19.496881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:19.496889 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:19.498337 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:19.498811 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:19.498811 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:19.498811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:19.498882 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:19.499130 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:19.499131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:19.499131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:19.499245 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:19.499313 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:19.499567 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 06:53:19.499646 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 06:53:19.499917 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:19.491552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:59157)
0926 06:53:19.491707 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:19.491746 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:19.492630 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:19.492835 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:19.492838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:19.493069 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:19.493253 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:19.493257 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.493950 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.493951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:19.494722 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:19.494725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.495031 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.495032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:19.495123 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:19.495524 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:19.495532 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:19.496766 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:19.498414 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:19.498421 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:19.498423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:19.498733 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:19.498952 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:19.498953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:19.498954 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:19.499064 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:19.499315 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:19.499321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:19.499447 (+ 126us) client_negotiation.cc:769] Sending connection context
0926 06:53:19.499546 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 06:53:19.499691 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260926 06:53:19.500381 9153 heartbeater.cc:347] Connected to a master server at 127.0.191.254:35967
I20260926 06:53:19.500468 9153 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:19.500630 9153 heartbeater.cc:511] Master 127.0.191.254:35967 requested a full tablet report, sending...
I20260926 06:53:19.501161 8962 ts_manager.cc:194] Registered new tserver with Master: ab0257bb3f6146ee8ddff40dd4f7bcc0 (127.0.191.193:45307)
I20260926 06:53:19.501811 8962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:59157
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:53:19.520277 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:35967
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:19.598518 9158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:19.598716 9158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:19.598738 9158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:19.599820 9158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:19.599854 9158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:19.599879 9158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:53:19.601281 9158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:35967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:19.601509 9158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:19.601691 9158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:19.606386 9166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:19.606452 9164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:19.606437 9163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:19.606585 9158 server_base.cc:1034] running on GCE node
I20260926 06:53:19.606935 9158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:19.607164 9158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:19.608311 9158 hybrid_clock.cc:648] HybridClock initialized: now 1790405599608301 us; error 30 us; skew 500 ppm
Sep 26 06:53:19 dist-test-slave-ws74 krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:19.610802 9158 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:53:19.611218 9158 webserver.cc:460] Webserver started at http://127.0.191.194:43743/ using document root <none> and password file <none>
I20260926 06:53:19.611393 9158 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:19.611433 9158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:19.611514 9158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:19.612469 9158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "9f2da9134b3f48439247d8df8c18567b"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "7f1ce8b5a1ffcec0606eb73adb4d44e7"
server_key_iv: "880c9f45022069bb57e6e620dd4e13fc"
server_key_version: "encryptionkey@0"
I20260926 06:53:19.612766 9158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "9f2da9134b3f48439247d8df8c18567b"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "7f1ce8b5a1ffcec0606eb73adb4d44e7"
server_key_iv: "880c9f45022069bb57e6e620dd4e13fc"
server_key_version: "encryptionkey@0"
I20260926 06:53:19.614058 9158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:53:19.614682 9173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.614866 9158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.614948 9158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "9f2da9134b3f48439247d8df8c18567b"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "7f1ce8b5a1ffcec0606eb73adb4d44e7"
server_key_iv: "880c9f45022069bb57e6e620dd4e13fc"
server_key_version: "encryptionkey@0"
I20260926 06:53:19.615012 9158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:19.624522 9158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:19.626622 9158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:19.626724 9158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:19.626922 9158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:19.627192 9158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:19.627247 9158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.627302 9158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:19.627333 9158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.632872 9158 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:45181
I20260926 06:53:19.632905 9286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:45181 every 8 connection(s)
I20260926 06:53:19.633201 9158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:53:19.634271 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 9158
I20260926 06:53:19.634377 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:53:19.634557 767 external_mini_cluster.cc:1428] Setting key 5536c29f8bd5e4ea4a449d10f1676ecd
Sep 26 06:53:19 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
I20260926 06:53:19.643394 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:19.633923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:40753 (local address 127.0.191.254:35967)
0926 06:53:19.633978 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:19.633980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:19.634169 (+ 189us) server_negotiation.cc:407] Connection header received
0926 06:53:19.635353 (+ 1184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:19.635354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:19.635366 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:19.635433 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:19.635982 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.636735 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.638257 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.638437 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.638487 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:19.640733 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:19.640734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:19.640735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:19.640742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:19.642075 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:19.642476 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:19.642477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:19.642478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:19.642531 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:19.642780 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:19.642781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:19.642782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:19.642890 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:19.642963 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:19.643222 (+ 259us) server_negotiation.cc:299] Negotiation successful
0926 06:53:19.643306 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 06:53:19.643600 9289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:19.633896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:40753)
0926 06:53:19.634056 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:19.634093 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:19.635243 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:19.635493 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:19.635496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:19.635723 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:19.635904 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:19.635907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.636821 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.636823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:19.638181 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:19.638183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.638512 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.638513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:19.638665 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:19.639138 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:19.639148 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:19.640643 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:19.642171 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:19.642177 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:19.642180 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:19.642396 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:19.642616 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:19.642617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:19.642617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:19.642715 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:19.642953 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:19.642959 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:19.643097 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 06:53:19.643201 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 06:53:19.643346 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260926 06:53:19.644016 9287 heartbeater.cc:347] Connected to a master server at 127.0.191.254:35967
I20260926 06:53:19.644120 9287 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:19.644289 9287 heartbeater.cc:511] Master 127.0.191.254:35967 requested a full tablet report, sending...
I20260926 06:53:19.644626 8962 ts_manager.cc:194] Registered new tserver with Master: 9f2da9134b3f48439247d8df8c18567b (127.0.191.194:45181)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
I20260926 06:53:19.645118 8962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:40753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:53:19.658377 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:35967
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:19.737586 9292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:19.737776 9292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:19.737799 9292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:19.738860 9292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:19.738891 9292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:19.738917 9292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:53:19.740310 9292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:35967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:19.740480 9292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:19.740707 9292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:53:19.745519 9292 server_base.cc:1034] running on GCE node
W20260926 06:53:19.745546 9297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:19.745489 9298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:19.745489 9300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:19.745914 9292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:19.746104 9292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:19.747228 9292 hybrid_clock.cc:648] HybridClock initialized: now 1790405599747216 us; error 31 us; skew 500 ppm
Sep 26 06:53:19 dist-test-slave-ws74 krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:19.749591 9292 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:53:19.749987 9292 webserver.cc:460] Webserver started at http://127.0.191.195:35795/ using document root <none> and password file <none>
I20260926 06:53:19.750160 9292 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:19.750200 9292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:19.750289 9292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:19.751184 9292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "7754979f58bc40c68ee24bbc82294d92"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "f54d5ab731519b92669c2de37b27d72b"
server_key_iv: "c227fefca66c68d97616ef9bb5c3cdbf"
server_key_version: "encryptionkey@0"
I20260926 06:53:19.751477 9292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "7754979f58bc40c68ee24bbc82294d92"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "f54d5ab731519b92669c2de37b27d72b"
server_key_iv: "c227fefca66c68d97616ef9bb5c3cdbf"
server_key_version: "encryptionkey@0"
I20260926 06:53:19.752653 9292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:53:19.753280 9307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.753448 9292 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.753528 9292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "7754979f58bc40c68ee24bbc82294d92"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "f54d5ab731519b92669c2de37b27d72b"
server_key_iv: "c227fefca66c68d97616ef9bb5c3cdbf"
server_key_version: "encryptionkey@0"
I20260926 06:53:19.753579 9292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:19.765910 9292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:19.767522 9292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:19.767627 9292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:19.767789 9292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:19.768016 9292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:19.768044 9292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.768064 9292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:19.768076 9292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.773581 9292 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:36861
I20260926 06:53:19.773659 9420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:36861 every 8 connection(s)
I20260926 06:53:19.773934 9292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:53:19 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:19.782287 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:19.774575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:57009 (local address 127.0.191.254:35967)
0926 06:53:19.774634 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:19.774636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:19.774825 (+ 189us) server_negotiation.cc:407] Connection header received
0926 06:53:19.775695 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:19.775696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:19.775706 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:19.775745 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:19.776286 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.776758 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.777755 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.777932 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.777978 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:19.780108 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:19.780110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:19.780110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:19.780117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:19.781247 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:19.781621 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:19.781622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:19.781622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:19.781668 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:19.781850 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:19.781850 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:19.781850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:19.781937 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:19.781983 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:19.782168 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 06:53:19.782220 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 06:53:19.782521 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 9292
I20260926 06:53:19.782648 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:53:19.782837 767 external_mini_cluster.cc:1428] Setting key df67709d1b7bb1b84cb607c9510dfd01
I20260926 06:53:19.783092 9423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:19.774543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:57009)
0926 06:53:19.774704 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:19.774750 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:19.775600 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:19.775805 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:19.775808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:19.776037 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:19.776213 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:19.776217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.776829 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.776830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:19.777683 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:19.777685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:19.777991 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:19.777992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:19.778083 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:19.778565 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:19.778573 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:19.780027 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:19.781310 (+ 1283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:19.781315 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:19.781317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:19.781566 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:19.781720 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:19.781721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:19.781721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:19.781811 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:19.781983 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:19.781986 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:19.782085 (+ 99us) client_negotiation.cc:769] Sending connection context
0926 06:53:19.782159 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 06:53:19.782273 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":98,"threads_started":1}
I20260926 06:53:19.783516 9421 heartbeater.cc:347] Connected to a master server at 127.0.191.254:35967
I20260926 06:53:19.783599 9421 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:19.783735 9421 heartbeater.cc:511] Master 127.0.191.254:35967 requested a full tablet report, sending...
I20260926 06:53:19.784097 8962 ts_manager.cc:194] Registered new tserver with Master: 7754979f58bc40c68ee24bbc82294d92 (127.0.191.195:36861)
I20260926 06:53:19.784426 8962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:57009
I20260926 06:53:19.794392 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab.
Entry for principal oryx/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab.
WARNING: no policy specified for oryx/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab.
Entry for principal oryx/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab.
WARNING: no policy specified for oryx/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab.
Entry for principal oryx/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab.
I20260926 06:53:19.856295 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 9024
I20260926 06:53:19.860787 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 9158
I20260926 06:53:19.864893 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 9292
I20260926 06:53:19.868899 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 8930
I20260926 06:53:19.873777 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:35967
--webserver_interface=127.0.191.254
--webserver_port=37969
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:35967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:19.953473 9430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:19.953662 9430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:19.953684 9430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:19.954762 9430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:19.954794 9430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:19.954809 9430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:19.954821 9430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:19.954834 9430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:19.956256 9430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:35967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:35967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=37969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:19.956435 9430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:19.956585 9430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:19.961014 9435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:19.961018 9436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:19.961046 9430 server_base.cc:1034] running on GCE node
W20260926 06:53:19.961023 9438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:19.961478 9430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:19.961761 9430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:19.962905 9430 hybrid_clock.cc:648] HybridClock initialized: now 1790405599962885 us; error 37 us; skew 500 ppm
Sep 26 06:53:19 dist-test-slave-ws74 krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:19.965236 9430 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:53:19.965684 9430 webserver.cc:460] Webserver started at http://127.0.191.254:37969/ using document root <none> and password file <none>
I20260926 06:53:19.965864 9430 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:19.965914 9430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:19.967090 9430 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:19.967703 9445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.967850 9430 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:19.967926 9430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "17d3cd84a01942e6b70283f101026ea8"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "1c21089c4b6a717d4d22772192d6866e"
server_key_iv: "1c6b0cdd606de55b15c17f538d748620"
server_key_version: "encryptionkey@0"
I20260926 06:53:19.968200 9430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:19.987337 9430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:19.989444 9430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:19.989542 9430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:19.993059 9430 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:35967
I20260926 06:53:19.993103 9497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:35967 every 8 connection(s)
I20260926 06:53:19.993607 9430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:19.994746 9498 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:53:19.995347 9498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8: Bootstrap starting.
I20260926 06:53:19.998030 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 9430
I20260926 06:53:19.998303 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:45307
--local_ip_for_outbound_sockets=127.0.191.193
--tserver_master_addrs=127.0.191.254:35967
--webserver_port=44145
--webserver_interface=127.0.191.193
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.191.193
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:53:20.003067 9498 log.cc:824] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:20.004546 9498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:53:20.004812 9498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8: Bootstrap complete.
I20260926 06:53:20.006970 9498 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d3cd84a01942e6b70283f101026ea8" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 35967 } }
I20260926 06:53:20.007192 9498 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17d3cd84a01942e6b70283f101026ea8, State: Initialized, Role: FOLLOWER
I20260926 06:53:20.007381 9498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d3cd84a01942e6b70283f101026ea8" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 35967 } }
I20260926 06:53:20.007480 9498 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:20.007548 9498 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:20.007599 9498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 1 FOLLOWER]: Advancing to term 2
I20260926 06:53:20.008558 9498 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d3cd84a01942e6b70283f101026ea8" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 35967 } }
I20260926 06:53:20.008668 9498 leader_election.cc:302] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17d3cd84a01942e6b70283f101026ea8; no voters:
I20260926 06:53:20.008857 9498 leader_election.cc:288] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 06:53:20.008886 9502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 06:53:20.008987 9502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [term 2 LEADER]: Becoming Leader. State: Replica: 17d3cd84a01942e6b70283f101026ea8, State: Running, Role: LEADER
I20260926 06:53:20.009104 9498 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:20.009133 9502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d3cd84a01942e6b70283f101026ea8" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 35967 } }
I20260926 06:53:20.009464 9503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "17d3cd84a01942e6b70283f101026ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d3cd84a01942e6b70283f101026ea8" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 35967 } } }
I20260926 06:53:20.009567 9503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:20.009460 9502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17d3cd84a01942e6b70283f101026ea8. Latest consensus state: current_term: 2 leader_uuid: "17d3cd84a01942e6b70283f101026ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d3cd84a01942e6b70283f101026ea8" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 35967 } } }
I20260926 06:53:20.009742 9502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:20.010281 9509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:20.010819 9509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:20.010946 9509 catalog_manager.cc:1259] Loaded cluster ID: 6e1705e77d88457b8b8aad7f0efc7335
I20260926 06:53:20.010982 9509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:53:20.012101 9509 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:20.012274 9509 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 17d3cd84a01942e6b70283f101026ea8: Loaded TSK: 0
I20260926 06:53:20.012498 9509 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 06:53:20.077621 9500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:20.077842 9500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:20.077881 9500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:20.079010 9500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:20.079077 9500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:20.079121 9500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:53:20.080499 9500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/oryx_127.0.191.193.keytab
--principal=oryx/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:45307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=44145
--tserver_master_addrs=127.0.191.254:35967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:20.080754 9500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:20.080963 9500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:20.086637 9524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:20.086746 9527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:20.086627 9525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:20.086989 9500 server_base.cc:1034] running on GCE node
I20260926 06:53:20.087175 9500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:20.087407 9500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:20.088557 9500 hybrid_clock.cc:648] HybridClock initialized: now 1790405600088546 us; error 30 us; skew 500 ppm
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405600, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:20.091185 9500 init.cc:377] Logged in from keytab as oryx/127.0.191.193@KRBTEST.COM (short username oryx)
I20260926 06:53:20.091699 9500 webserver.cc:460] Webserver started at http://127.0.191.193:44145/ using document root <none> and password file <none>
I20260926 06:53:20.091964 9500 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:20.092060 9500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:20.093679 9500 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:53:20.094290 9534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:20.094478 9500 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:53:20.094564 9500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "ab0257bb3f6146ee8ddff40dd4f7bcc0"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "fdb3de5932b4f447002529e0c7893b4c"
server_key_iv: "2f4bf18d268d36d5007ed799eca831ee"
server_key_version: "encryptionkey@0"
I20260926 06:53:20.094960 9500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:20.129590 9500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:20.132218 9500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:20.132360 9500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:20.132637 9500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:20.132962 9500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:20.133003 9500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:20.133033 9500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:20.133049 9500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:20.140287 9500 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:45307
I20260926 06:53:20.140329 9647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:45307 every 8 connection(s)
I20260926 06:53:20.140707 9500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:53:20.142997 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 9500
I20260926 06:53:20.143322 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:45181
--local_ip_for_outbound_sockets=127.0.191.194
--tserver_master_addrs=127.0.191.254:35967
--webserver_port=43743
--webserver_interface=127.0.191.194
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.191.194
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:20.149971 9650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:20.141449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:39373)
0926 06:53:20.141615 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:20.141657 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:20.142119 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:20.142476 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:20.142479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:20.143040 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:20.143294 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.143299 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.144876 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.144878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.146018 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.146021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.146847 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.146848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.146941 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.147457 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:20.147465 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:20.149840 (+ 2375us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
W20260926 06:53:20.150197 9651 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:20.141495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:39373 (local address 127.0.191.254:35967)
0926 06:53:20.141697 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:20.141704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:20.141734 (+ 30us) server_negotiation.cc:407] Connection header received
0926 06:53:20.142275 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:20.142280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:20.142320 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:20.142428 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:20.143413 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.144782 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.146107 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.146761 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.146886 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.149996 (+ 3110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:39373: BlockingRecv error: recv got EOF from 127.0.191.193:39373 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
W20260926 06:53:20.150202 9648 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:35967 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.191.254:35967: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:20.154531 9650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:20.150422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:50427)
0926 06:53:20.150568 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:20.150581 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:20.150634 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:20.150767 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:20.150768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:20.150890 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:20.150926 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.150928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.151525 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.151526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.152023 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.152024 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.152357 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.152358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.152418 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.152859 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:20.152862 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:20.154454 (+ 1592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
W20260926 06:53:20.154580 9651 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:20.150465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:50427 (local address 127.0.191.254:35967)
0926 06:53:20.150524 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:20.150526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:20.150622 (+ 96us) server_negotiation.cc:407] Connection header received
0926 06:53:20.150691 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:20.150692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:20.150703 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:20.150767 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:20.151003 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.151428 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.152110 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.152291 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.152359 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.154519 (+ 2160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:50427: BlockingRecv error: recv got EOF from 127.0.191.193:50427 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:20.158420 9650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:20.154759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:51991)
0926 06:53:20.154801 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:20.154811 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:20.154846 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:20.154970 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:20.154971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:20.155081 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:20.155103 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.155105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.155677 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.155678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.156142 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.156143 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.156499 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.156500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.156550 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.156931 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:20.156935 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:20.158356 (+ 1421us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 06:53:20.158466 9651 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:20.154792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:51991 (local address 127.0.191.254:35967)
0926 06:53:20.154824 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:20.154826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:20.154835 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:20.154902 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:20.154903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:20.154913 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:20.154966 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:20.155179 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.155608 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.156230 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.156408 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.156498 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.158414 (+ 1916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:51991: BlockingRecv error: recv got EOF from 127.0.191.193:51991 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 06:53:20.158636 9648 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 06:53:20.224328 9652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:20.224507 9652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:20.224529 9652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:20.225740 9652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:20.225780 9652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:20.225826 9652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:53:20.227221 9652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/oryx_127.0.191.194.keytab
--principal=oryx/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:45181
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=43743
--tserver_master_addrs=127.0.191.254:35967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:20.227439 9652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:20.227643 9652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:20.232260 9659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:20.232235 9658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:20.232236 9661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:20.232579 9652 server_base.cc:1034] running on GCE node
I20260926 06:53:20.232785 9652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:20.233016 9652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:20.234162 9652 hybrid_clock.cc:648] HybridClock initialized: now 1790405600234143 us; error 26 us; skew 500 ppm
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405600, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:20.237190 9652 init.cc:377] Logged in from keytab as oryx/127.0.191.194@KRBTEST.COM (short username oryx)
I20260926 06:53:20.237597 9652 webserver.cc:460] Webserver started at http://127.0.191.194:43743/ using document root <none> and password file <none>
I20260926 06:53:20.237792 9652 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:20.237841 9652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:20.238996 9652 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:20.239465 9668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:20.239629 9652 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:20.239717 9652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "9f2da9134b3f48439247d8df8c18567b"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "7f1ce8b5a1ffcec0606eb73adb4d44e7"
server_key_iv: "880c9f45022069bb57e6e620dd4e13fc"
server_key_version: "encryptionkey@0"
I20260926 06:53:20.240034 9652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:20.248760 9652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:20.250507 9652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:20.250651 9652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:20.250854 9652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:20.251132 9652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:20.251178 9652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:20.251215 9652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:20.251241 9652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:20.256711 9652 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:45181
I20260926 06:53:20.256757 9781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:45181 every 8 connection(s)
I20260926 06:53:20.257107 9652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:53:20.258095 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 9652
I20260926 06:53:20.258366 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:36861
--local_ip_for_outbound_sockets=127.0.191.195
--tserver_master_addrs=127.0.191.254:35967
--webserver_port=35795
--webserver_interface=127.0.191.195
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.191.195
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 06:53:20.264463 9651 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:20.257726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:50461 (local address 127.0.191.254:35967)
0926 06:53:20.257798 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:20.257800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:20.258001 (+ 201us) server_negotiation.cc:407] Connection header received
0926 06:53:20.258591 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:20.258593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:20.258606 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:20.258641 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:20.259588 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.260124 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.261098 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.261305 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.261374 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.264417 (+ 3043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:50461: BlockingRecv error: recv got EOF from 127.0.191.194:50461 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 06:53:20.264533 9784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:20.257739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:50461)
0926 06:53:20.257891 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:20.257922 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:20.258466 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:20.258782 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:20.258785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:20.259292 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:20.259483 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.259487 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.260224 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.260224 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.261001 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.261003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.261411 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.261412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.261524 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.261992 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:20.261999 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:20.264318 (+ 2319us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
W20260926 06:53:20.264745 9782 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:35967 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.191.254:35967: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:20.269073 9784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:20.264967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:47911)
0926 06:53:20.265025 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:20.265034 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:20.265068 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:20.265193 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:20.265194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:20.265318 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:20.265355 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.265356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.266006 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.266008 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.266502 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.266503 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.266848 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.266849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.266918 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.267332 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:20.267336 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:20.268999 (+ 1663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 06:53:20.269122 9651 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:20.265021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:47911 (local address 127.0.191.254:35967)
0926 06:53:20.265071 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:20.265073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:20.265080 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:53:20.265119 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:20.265120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:20.265129 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:20.265190 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:20.265489 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.265934 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.266578 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.266770 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.266847 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.269062 (+ 2215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:47911: BlockingRecv error: recv got EOF from 127.0.191.194:47911 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:20.273903 9786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:20.269298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:59975)
0926 06:53:20.269438 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:20.269448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:20.269503 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:20.269628 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:20.269629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:20.269756 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:20.269800 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.269800 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.270629 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.270629 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.271176 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.271178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.271608 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.271609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.271652 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.272150 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:20.272154 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:20.273830 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":115,"threads_started":1}
W20260926 06:53:20.273931 9651 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:20.269341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:59975 (local address 127.0.191.254:35967)
0926 06:53:20.269408 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:20.269410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:20.269502 (+ 92us) server_negotiation.cc:407] Connection header received
0926 06:53:20.269560 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:20.269560 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:20.269569 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:20.269621 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:20.269857 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.270567 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.271237 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.271532 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.271596 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.273895 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:59975: BlockingRecv error: recv got EOF from 127.0.191.194:59975 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260926 06:53:20.274008 9782 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 06:53:20.339985 9785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:20.340166 9785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:20.340188 9785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:20.341362 9785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:20.341415 9785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:20.341440 9785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:53:20.342834 9785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/krb5kdc/oryx_127.0.191.195.keytab
--principal=oryx/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:36861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=35795
--tserver_master_addrs=127.0.191.254:35967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:20.343001 9785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:20.343182 9785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:20.347820 9791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:20.347846 9794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:20.347816 9792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:20.348031 9785 server_base.cc:1034] running on GCE node
I20260926 06:53:20.348210 9785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:20.348448 9785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:20.349641 9785 hybrid_clock.cc:648] HybridClock initialized: now 1790405600349626 us; error 26 us; skew 500 ppm
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405600, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:20.352160 9785 init.cc:377] Logged in from keytab as oryx/127.0.191.195@KRBTEST.COM (short username oryx)
I20260926 06:53:20.352569 9785 webserver.cc:460] Webserver started at http://127.0.191.195:35795/ using document root <none> and password file <none>
I20260926 06:53:20.352753 9785 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:20.352803 9785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:20.354034 9785 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:53:20.354547 9801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:20.354709 9785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:20.354796 9785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "7754979f58bc40c68ee24bbc82294d92"
format_stamp: "Formatted at 2026-09-26 06:53:19 on dist-test-slave-ws74"
server_key: "f54d5ab731519b92669c2de37b27d72b"
server_key_iv: "c227fefca66c68d97616ef9bb5c3cdbf"
server_key_version: "encryptionkey@0"
I20260926 06:53:20.355089 9785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:20.366415 9785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:20.368052 9785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:20.368193 9785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:20.368383 9785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:20.368649 9785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:20.368681 9785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:20.368716 9785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:20.368749 9785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:20.374619 9785 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:36861
I20260926 06:53:20.374682 9914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:36861 every 8 connection(s)
I20260926 06:53:20.375003 9785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 06:53:20.381914 9651 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:20.375646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:48343 (local address 127.0.191.254:35967)
0926 06:53:20.375702 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:20.375704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:20.375903 (+ 199us) server_negotiation.cc:407] Connection header received
0926 06:53:20.376401 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:20.376402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:20.376414 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:20.376459 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:20.377359 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.377868 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.378835 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.379032 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.379109 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.381847 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:48343: BlockingRecv error: recv got EOF from 127.0.191.195:48343 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 06:53:20.381968 9917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:20.375643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:48343)
0926 06:53:20.375797 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:20.375830 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:20.376295 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:20.376532 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:20.376536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:20.377087 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:20.377266 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.377271 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.377969 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.377970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.378741 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.378743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.379117 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.379118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.379244 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.379704 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:20.379711 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:20.381777 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
W20260926 06:53:20.382136 9915 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:35967 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.191.254:35967: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
I20260926 06:53:20.382853 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 9785
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:20.386608 9917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:20.382402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:54257)
0926 06:53:20.382445 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:20.382454 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:20.382488 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:20.382654 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:20.382655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:20.382787 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:20.382824 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.382826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.383607 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.383608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.384087 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.384088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.384445 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.384446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.384512 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.384913 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:20.384918 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:20.386534 (+ 1616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 06:53:20.386655 9651 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:20.382531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:54257 (local address 127.0.191.254:35967)
0926 06:53:20.382566 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:20.382568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:20.382573 (+ 5us) server_negotiation.cc:407] Connection header received
0926 06:53:20.382588 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:20.382589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:20.382596 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:20.382651 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:20.382893 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.383526 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.384173 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.384368 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.384445 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.386595 (+ 2150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:54257: BlockingRecv error: recv got EOF from 127.0.191.195:54257 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:20.390084 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:20.383241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46320 (local address 127.0.191.254:35967)
0926 06:53:20.383382 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:20.383384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:20.383469 (+ 85us) server_negotiation.cc:407] Connection header received
0926 06:53:20.383571 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:20.383572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:20.383583 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:20.383646 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:20.384228 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.384694 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.385461 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.385701 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.385760 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.386900 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:20.386903 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:20.386904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:20.386932 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:20.388651 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:20.389053 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:20.389056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:20.389056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:20.389153 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:20.389347 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:20.389347 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:20.389348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:20.389607 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:20.389748 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:20.389865 (+ 117us) server_negotiation.cc:299] Negotiation successful
0926 06:53:20.389972 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
Sep 26 06:53:20 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:20.390985 9917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:20.387064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:57575)
0926 06:53:20.387104 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:20.387114 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:20.387148 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:20.387248 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:20.387249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:20.387377 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:20.387397 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.387398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.387995 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.387996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.388465 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:20.388466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.388821 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.388822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:20.388871 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.389267 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:20.389271 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:20.390926 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 06:53:20.391032 9651 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:20.387118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:57575 (local address 127.0.191.254:35967)
0926 06:53:20.387156 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:20.387157 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:20.387166 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:20.387191 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:20.387191 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:20.387199 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:20.387246 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:20.387467 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.387914 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.388551 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:20.388746 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:20.388820 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:20.390986 (+ 2166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:57575: BlockingRecv error: recv got EOF from 127.0.191.195:57575 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 06:53:20.391151 9915 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 06:53:21 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:21 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:21.163130 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:21.158967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:50545)
0926 06:53:21.159109 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:21.159119 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:21.159179 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:21.159315 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:21.159316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:21.159459 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:21.159524 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:21.159526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:21.160177 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:21.160177 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:21.160704 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:21.160705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:21.160993 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:21.160994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:21.161039 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:21.161464 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:21.161468 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:21.163014 (+ 1546us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
W20260926 06:53:21.163187 9921 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:21.159024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:50545 (local address 127.0.191.254:35967)
0926 06:53:21.159119 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:21.159121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:21.159184 (+ 63us) server_negotiation.cc:407] Connection header received
0926 06:53:21.159242 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:21.159242 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:21.159252 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:21.159299 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:21.159585 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:21.160116 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:21.160766 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:21.160938 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:21.160983 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:21.163114 (+ 2131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:50545: BlockingRecv error: recv got EOF from 127.0.191.193:50545 (error 108)
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":42,"threads_started":1}
Sep 26 06:53:21 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:21 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:21.278574 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:21.274404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:55801)
0926 06:53:21.274517 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:21.274528 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:21.274587 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:21.274700 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:21.274701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:21.274860 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:21.274906 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:21.274908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:21.275498 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:21.275499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:21.276010 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:21.276012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:21.276302 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:21.276302 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:21.276340 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:21.276749 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:21.276753 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:21.278519 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":66,"threads_started":1}
W20260926 06:53:21.278702 9921 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:21.274481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:55801 (local address 127.0.191.254:35967)
0926 06:53:21.274528 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:21.274530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:21.274576 (+ 46us) server_negotiation.cc:407] Connection header received
0926 06:53:21.274628 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:21.274629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:21.274637 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:21.274689 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:21.274964 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:21.275440 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:21.276072 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:21.276247 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:21.276294 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:21.278623 (+ 2329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:55801: BlockingRecv error: recv got EOF from 127.0.191.194:55801 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 26 06:53:21 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:21 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:21.395875 9923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:21.391501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:60873)
0926 06:53:21.391622 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:21.391633 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:21.391687 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:21.391819 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:21.391820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:21.391989 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:21.392032 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:21.392034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:21.392671 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:21.392672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:21.393214 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:21.393215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:21.393648 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:21.393649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:21.393714 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:21.394124 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:21.394128 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:21.395777 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
W20260926 06:53:21.395910 9921 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:21.391622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:60873 (local address 127.0.191.254:35967)
0926 06:53:21.391682 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:21.391684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:21.391692 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:53:21.391736 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:21.391737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:21.391751 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:21.391814 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:21.392109 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:21.392595 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:21.393323 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:21.393570 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:21.393647 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:21.395850 (+ 2203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:60873: BlockingRecv error: recv got EOF from 127.0.191.195:60873 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 26 06:53:22 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:22 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:22.167816 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:22.163697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:57469)
0926 06:53:22.163821 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:22.163831 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:22.163878 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:22.164001 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:22.164002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:22.164151 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:22.164198 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:22.164200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:22.164808 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:22.164808 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:22.165333 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:22.165334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:22.165695 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:22.165695 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:22.165736 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:22.166151 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:22.166155 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:22.167760 (+ 1605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
W20260926 06:53:22.167908 9925 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:22.163763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:57469 (local address 127.0.191.254:35967)
0926 06:53:22.163877 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:22.163879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:22.163888 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:22.163919 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:22.163919 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:22.163930 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:22.163990 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:22.164256 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:22.164746 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:22.165433 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:22.165636 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:22.165687 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:22.167838 (+ 2151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:57469: BlockingRecv error: recv got EOF from 127.0.191.193:57469 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":67,"threads_started":1}
Sep 26 06:53:22 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:22 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:22.283272 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:22.279029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:43057)
0926 06:53:22.279157 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:22.279173 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:22.279240 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:22.279360 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:22.279361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:22.279512 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:22.279558 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:22.279559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:22.280147 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:22.280148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:22.280660 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:22.280661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:22.280966 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:22.280967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:22.281004 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:22.281535 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:22.281540 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:22.283214 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
W20260926 06:53:22.283381 9925 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:22.279096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:43057 (local address 127.0.191.254:35967)
0926 06:53:22.279142 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:22.279144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:22.279229 (+ 85us) server_negotiation.cc:407] Connection header received
0926 06:53:22.279281 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:22.279282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:22.279291 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:22.279341 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:22.279612 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:22.280090 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:22.280723 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:22.280909 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:22.280959 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:22.283308 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:43057: BlockingRecv error: recv got EOF from 127.0.191.194:43057 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 26 06:53:22 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:22 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:22.400458 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:22.396282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:47963)
0926 06:53:22.396407 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:22.396417 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:22.396461 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:22.396584 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:22.396585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:22.396745 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:22.396791 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:22.396792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:22.397426 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:22.397426 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:22.397945 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:22.397947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:22.398236 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:22.398237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:22.398273 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:22.398690 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:22.398694 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:22.400403 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":52,"threads_started":1}
W20260926 06:53:22.400553 9925 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:22.396422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:47963 (local address 127.0.191.254:35967)
0926 06:53:22.396485 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:22.396487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:22.396494 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:53:22.396513 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:22.396514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:22.396524 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:22.396570 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:22.396850 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:22.397327 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:22.398008 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:22.398181 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:22.398229 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:22.400489 (+ 2260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:47963: BlockingRecv error: recv got EOF from 127.0.191.195:47963 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 26 06:53:23 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:23 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:23.172791 9928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:23.168379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:37765)
0926 06:53:23.168496 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:23.168507 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:23.168564 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:23.168710 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:23.168711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:23.168863 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:23.168911 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:23.168912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:23.169580 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:23.169580 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:23.170100 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:23.170101 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:23.170444 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:23.170445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:23.170488 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:23.170897 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:23.170901 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:23.172683 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
W20260926 06:53:23.172850 9929 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:23.168477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:37765 (local address 127.0.191.254:35967)
0926 06:53:23.168599 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:23.168601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:23.168609 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:53:23.168628 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:23.168629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:23.168639 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:23.168699 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:23.168971 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:23.169523 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:23.170183 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:23.170387 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:23.170436 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:23.172774 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:37765: BlockingRecv error: recv got EOF from 127.0.191.193:37765 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
Sep 26 06:53:23 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:23 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:23.288158 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:23.283737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:37241)
0926 06:53:23.283873 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:23.283884 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:23.283948 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:23.284069 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:23.284069 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:23.284220 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:23.284276 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:23.284277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:23.284888 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:23.284889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:23.285574 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:23.285577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:23.285897 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:23.285898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:23.285936 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:23.286351 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:23.286355 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:23.288063 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
W20260926 06:53:23.288206 9929 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:23.283817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:37241 (local address 127.0.191.254:35967)
0926 06:53:23.283859 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:23.283861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:23.283929 (+ 68us) server_negotiation.cc:407] Connection header received
0926 06:53:23.283992 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:23.283993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:23.284003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:23.284053 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:23.284335 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:23.284824 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:23.285640 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:23.285832 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:23.285886 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:23.288142 (+ 2256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:37241: BlockingRecv error: recv got EOF from 127.0.191.194:37241 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 26 06:53:23 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:23 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:23.405402 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:23.400893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:50313)
0926 06:53:23.401046 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:23.401058 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:23.401122 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:23.401256 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:23.401256 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:23.401441 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:23.401488 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:23.401489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:23.402214 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:23.402215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:23.402725 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:23.402727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:23.403103 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:23.403103 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:23.403140 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:23.403591 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:23.403595 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:23.405317 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":105,"threads_started":1}
W20260926 06:53:23.405553 9929 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:23.400978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:50313 (local address 127.0.191.254:35967)
0926 06:53:23.401034 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:23.401036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:23.401126 (+ 90us) server_negotiation.cc:407] Connection header received
0926 06:53:23.401164 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:23.401165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:23.401177 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:23.401225 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:23.401537 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:23.402125 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:23.402834 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:23.403018 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:23.403098 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:23.405482 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:50313: BlockingRecv error: recv got EOF from 127.0.191.195:50313 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 26 06:53:24 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:24 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:24.177570 9932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:24.173291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:43509)
0926 06:53:24.173457 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:24.173469 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:24.173560 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:24.173737 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:24.173738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:24.173909 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:24.173959 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:24.173961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:24.174583 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:24.174583 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:24.175084 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:24.175086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:24.175389 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:24.175389 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:24.175427 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:24.175855 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:24.175859 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:24.177515 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
W20260926 06:53:24.177685 9933 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:24.173326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:43509 (local address 127.0.191.254:35967)
0926 06:53:24.173465 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:24.173481 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:24.173525 (+ 44us) server_negotiation.cc:407] Connection header received
0926 06:53:24.173632 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:24.173633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:24.173645 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:24.173718 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:24.174023 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:24.174503 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:24.175147 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:24.175329 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:24.175377 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:24.177604 (+ 2227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:43509: BlockingRecv error: recv got EOF from 127.0.191.193:43509 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
Sep 26 06:53:24 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:24 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:24.298013 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:24.288735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:39495)
0926 06:53:24.288926 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:24.288937 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:24.289026 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:24.289172 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:24.289173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:24.289356 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:24.289425 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:24.289426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:24.290084 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:24.290086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:24.290613 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:24.290614 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:24.291044 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:24.291045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:24.291103 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:24.296218 (+ 5115us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:24.296223 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:24.297956 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":86,"threads_started":1}
W20260926 06:53:24.298149 9933 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:24.288812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:39495 (local address 127.0.191.254:35967)
0926 06:53:24.288893 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:24.288896 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:24.289010 (+ 114us) server_negotiation.cc:407] Connection header received
0926 06:53:24.289081 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:24.289082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:24.289095 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:24.289166 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:24.289513 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:24.290005 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:24.290705 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:24.290968 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:24.291047 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:24.298068 (+ 7021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:39495: BlockingRecv error: recv got EOF from 127.0.191.194:39495 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 26 06:53:24 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:24 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:24.410461 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:24.405869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:39381)
0926 06:53:24.406009 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:24.406026 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:24.406078 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:24.406203 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:24.406204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:24.406361 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:24.406409 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:24.406410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:24.407022 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:24.407022 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:24.407519 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:24.407520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:24.407827 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:24.407828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:24.407866 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:24.408358 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:24.408363 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:24.410367 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":95,"threads_started":1}
W20260926 06:53:24.410494 9933 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:24.405918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:39381 (local address 127.0.191.254:35967)
0926 06:53:24.405986 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:24.405988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:24.406077 (+ 89us) server_negotiation.cc:407] Connection header received
0926 06:53:24.406132 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:24.406133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:24.406146 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:24.406186 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:24.406464 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:24.406961 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:24.407581 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:24.407766 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:24.407816 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:24.410441 (+ 2625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:39381: BlockingRecv error: recv got EOF from 127.0.191.195:39381 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 26 06:53:25 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:25 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:25.182293 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:25.178025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:39143)
0926 06:53:25.178168 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:25.178178 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:25.178229 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:25.178352 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:25.178353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:25.178507 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:25.178556 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:25.178558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:25.179175 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:25.179176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:25.179679 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:25.179680 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:25.179995 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:25.179995 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:25.180032 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:25.180540 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:25.180544 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:25.182228 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
W20260926 06:53:25.182394 9937 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:25.178111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:39143 (local address 127.0.191.254:35967)
0926 06:53:25.178231 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:25.178232 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:25.178258 (+ 26us) server_negotiation.cc:407] Connection header received
0926 06:53:25.178277 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:25.178277 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:25.178287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:25.178335 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:25.178620 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:25.179113 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:25.179752 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:25.179934 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:25.179982 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:25.182322 (+ 2340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:39143: BlockingRecv error: recv got EOF from 127.0.191.193:39143 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":48,"threads_started":1}
Sep 26 06:53:25 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:25 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:25.303025 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:25.298528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:51995)
0926 06:53:25.298662 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:25.298673 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:25.298769 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:25.298907 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:25.298908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:25.299077 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:25.299123 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:25.299125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:25.299842 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:25.299843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:25.300364 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:25.300366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:25.300740 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:25.300741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:25.300801 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:25.301231 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:25.301236 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:25.302931 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":56,"threads_started":1}
W20260926 06:53:25.303082 9937 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:25.298667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:51995 (local address 127.0.191.254:35967)
0926 06:53:25.298727 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:25.298729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:25.298739 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:53:25.298831 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:25.298832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:25.298841 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:25.298891 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:25.299205 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:25.299744 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:25.300449 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:25.300647 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:25.300732 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:25.303007 (+ 2275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:51995: BlockingRecv error: recv got EOF from 127.0.191.194:51995 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 26 06:53:25 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:25 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:25.415197 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:25.411066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:48083)
0926 06:53:25.411192 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:25.411202 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:25.411273 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:25.411376 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:25.411377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:25.411533 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:25.411579 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:25.411580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:25.412183 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:25.412183 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:25.412688 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:25.412689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:25.412993 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:25.412994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:25.413032 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:25.413463 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:25.413468 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:25.415143 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
W20260926 06:53:25.415330 9937 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:25.411105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:48083 (local address 127.0.191.254:35967)
0926 06:53:25.411150 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:25.411153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:25.411275 (+ 122us) server_negotiation.cc:407] Connection header received
0926 06:53:25.411314 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:25.411315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:25.411326 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:25.411365 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:25.411634 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:25.412118 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:25.412760 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:25.412932 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:25.412980 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:25.415250 (+ 2270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:48083: BlockingRecv error: recv got EOF from 127.0.191.195:48083 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 26 06:53:26 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:26 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:26.187425 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:26.182773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:55597)
0926 06:53:26.182915 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:26.182925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:26.182972 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:26.183202 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:26.183203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:26.183359 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:26.183405 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:26.183406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:26.184018 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:26.184019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:26.184653 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:26.184669 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:26.185085 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:26.185086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:26.185126 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:26.185653 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:26.185665 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:26.187369 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
W20260926 06:53:26.187531 9941 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:26.182944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:55597 (local address 127.0.191.254:35967)
0926 06:53:26.183078 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:26.183080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:26.183092 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:53:26.183112 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:26.183112 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:26.183122 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:26.183185 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:26.183472 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:26.183956 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:26.184756 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:26.185025 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:26.185076 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:26.187455 (+ 2379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:55597: BlockingRecv error: recv got EOF from 127.0.191.193:55597 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
Sep 26 06:53:26 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:26 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:26.307905 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:26.303553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:50617)
0926 06:53:26.303676 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:26.303695 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:26.303751 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:26.303875 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:26.303876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:26.304036 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:26.304085 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:26.304087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:26.304695 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:26.304695 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:26.305209 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:26.305210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:26.305581 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:26.305582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:26.305625 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:26.306047 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:26.306052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:26.307843 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
W20260926 06:53:26.308033 9941 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:26.303628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:50617 (local address 127.0.191.254:35967)
0926 06:53:26.303688 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:26.303690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:26.303753 (+ 63us) server_negotiation.cc:407] Connection header received
0926 06:53:26.303803 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:26.303804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:26.303814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:26.303854 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:26.304146 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:26.304631 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:26.305283 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:26.305518 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:26.305569 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:26.307934 (+ 2365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:50617: BlockingRecv error: recv got EOF from 127.0.191.194:50617 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 26 06:53:26 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:26 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:26.420288 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:26.415690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:45169)
0926 06:53:26.415860 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:26.415887 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:26.415933 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:26.416068 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:26.416069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:26.416229 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:26.416297 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:26.416299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:26.416893 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:26.416893 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:26.417429 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:26.417430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:26.417736 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:26.417737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:26.417783 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:26.418181 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:26.418185 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:26.420228 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":129,"threads_started":1}
W20260926 06:53:26.420425 9941 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:26.415865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:45169 (local address 127.0.191.254:35967)
0926 06:53:26.415917 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:26.415919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:26.415928 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:26.415994 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:26.415994 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:26.416005 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:26.416047 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:26.416361 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:26.416832 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:26.417499 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:26.417674 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:26.417725 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:26.420344 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:45169: BlockingRecv error: recv got EOF from 127.0.191.195:45169 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 26 06:53:27 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:27 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:27.192440 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:27.187919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:36515)
0926 06:53:27.188051 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:27.188063 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:27.188127 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:27.188336 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:27.188337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:27.188496 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:27.188557 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:27.188558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:27.189187 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:27.189188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:27.189745 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:27.189747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:27.190062 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:27.190063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:27.190106 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:27.190533 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:27.190538 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:27.192384 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
W20260926 06:53:27.192590 9945 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:27.188093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:36515 (local address 127.0.191.254:35967)
0926 06:53:27.188219 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:27.188221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:27.188228 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:53:27.188248 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:27.188248 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:27.188258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:27.188318 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:27.188619 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:27.189109 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:27.189812 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:27.189994 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:27.190046 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:27.192503 (+ 2457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:36515: BlockingRecv error: recv got EOF from 127.0.191.193:36515 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
Sep 26 06:53:27 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:27 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:27.313488 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:27.308477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:39623)
0926 06:53:27.308612 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:27.308625 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:27.308691 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:27.308858 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:27.308859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:27.309038 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:27.309083 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:27.309084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:27.309881 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:27.309882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:27.310499 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:27.310501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:27.310853 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:27.310854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:27.310910 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:27.311460 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:27.311465 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:27.313351 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
W20260926 06:53:27.313558 9945 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:27.308586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:39623 (local address 127.0.191.254:35967)
0926 06:53:27.308643 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:27.308646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:27.308671 (+ 25us) server_negotiation.cc:407] Connection header received
0926 06:53:27.308776 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:27.308777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:27.308788 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:27.308851 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:27.309160 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:27.309763 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:27.310582 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:27.310783 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:27.310856 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:27.313489 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:39623: BlockingRecv error: recv got EOF from 127.0.191.194:39623 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 26 06:53:27 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:27 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:27.425712 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:27.420810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:52281)
0926 06:53:27.421000 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:27.421012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:27.421136 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:27.421326 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:27.421327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:27.421519 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:27.421567 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:27.421569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:27.422257 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:27.422258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:27.422835 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:27.422837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:27.423238 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:27.423239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:27.423296 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:27.423717 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:27.423721 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:27.425655 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":90,"threads_started":1}
W20260926 06:53:27.425832 9945 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:27.420891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:52281 (local address 127.0.191.254:35967)
0926 06:53:27.420984 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:27.420986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:27.421118 (+ 132us) server_negotiation.cc:407] Connection header received
0926 06:53:27.421229 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:27.421230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:27.421248 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:27.421319 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:27.421649 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:27.422137 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:27.422922 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:27.423135 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:27.423240 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:27.425736 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:52281: BlockingRecv error: recv got EOF from 127.0.191.195:52281 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 06:53:28 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:28 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:28.197501 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:28.193120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:34839)
0926 06:53:28.193230 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:28.193240 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:28.193293 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:28.193457 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:28.193458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:28.193664 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:28.193710 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:28.193712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:28.194317 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:28.194318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:28.194816 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:28.194818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:28.195117 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:28.195118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:28.195155 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:28.195652 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:28.195657 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:28.197440 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
W20260926 06:53:28.197558 9949 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:28.193203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:34839 (local address 127.0.191.254:35967)
0926 06:53:28.193314 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:28.193317 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:28.193327 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:53:28.193350 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:28.193351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:28.193363 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:28.193447 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:28.193779 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:28.194254 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:28.194881 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:28.195059 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:28.195106 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:28.197484 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:34839: BlockingRecv error: recv got EOF from 127.0.191.193:34839 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":71,"threads_started":1}
Sep 26 06:53:28 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:28 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:28.318190 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:28.313889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:46129)
0926 06:53:28.314031 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:28.314042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:28.314106 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:28.314223 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:28.314224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:28.314380 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:28.314428 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:28.314430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:28.315151 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:28.315151 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:28.315660 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:28.315661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:28.315957 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:28.315958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:28.315996 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:28.316409 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:28.316413 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:28.318133 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
W20260926 06:53:28.318321 9949 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:28.313949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:46129 (local address 127.0.191.254:35967)
0926 06:53:28.314009 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:28.314011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:28.314107 (+ 96us) server_negotiation.cc:407] Connection header received
0926 06:53:28.314145 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:28.314146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:28.314155 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:28.314210 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:28.314484 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:28.315090 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:28.315723 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:28.315897 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:28.315943 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:28.318216 (+ 2273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:46129: BlockingRecv error: recv got EOF from 127.0.191.194:46129 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 26 06:53:28 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:28 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:28.431010 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:28.426154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:60781)
0926 06:53:28.426309 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:28.426321 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:28.426389 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:28.426519 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:28.426520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:28.426678 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:28.426726 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:28.426728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:28.427410 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:28.427410 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:28.427925 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:28.427926 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:28.428399 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:28.428400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:28.428454 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:28.429131 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:28.429136 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:28.430954 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":108,"threads_started":1}
W20260926 06:53:28.431125 9949 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:28.426237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:60781 (local address 127.0.191.254:35967)
0926 06:53:28.426299 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:28.426301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:28.426387 (+ 86us) server_negotiation.cc:407] Connection header received
0926 06:53:28.426430 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:28.426431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:28.426442 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:28.426493 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:28.426846 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:28.427332 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:28.427997 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:28.428197 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:28.428250 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:28.431033 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:60781: BlockingRecv error: recv got EOF from 127.0.191.195:60781 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 26 06:53:29 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:29 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:29.202286 9952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:29.197891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:36357)
0926 06:53:29.198030 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:29.198041 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:29.198097 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:29.198217 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:29.198218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:29.198431 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:29.198477 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:29.198479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:29.199146 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:29.199147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:29.199731 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:29.199733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:29.200039 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:29.200039 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:29.200115 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:29.200535 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:29.200539 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:29.202228 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
W20260926 06:53:29.202404 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:29.197976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:36357 (local address 127.0.191.254:35967)
0926 06:53:29.198071 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:29.198073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:29.198084 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:53:29.198149 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:29.198150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:29.198160 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:29.198230 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:29.198538 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:29.199087 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:29.199793 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:29.199975 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:29.200024 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:29.202323 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:36357: BlockingRecv error: recv got EOF from 127.0.191.193:36357 (error 108)
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":49,"threads_started":1}
Sep 26 06:53:29 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:29 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:29.328390 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:29.318623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:59223)
0926 06:53:29.318808 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:29.318822 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:29.318904 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:29.319065 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:29.319065 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:29.319250 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:29.319296 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:29.319298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:29.319979 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:29.319980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:29.320516 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:29.320518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:29.320881 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:29.320882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:29.320941 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:29.326457 (+ 5516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:29.326462 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:29.328278 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
W20260926 06:53:29.328424 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:29.318731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:59223 (local address 127.0.191.254:35967)
0926 06:53:29.318837 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:29.318840 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:29.318898 (+ 58us) server_negotiation.cc:407] Connection header received
0926 06:53:29.318981 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:29.318982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:29.318994 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:29.319051 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:29.319384 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:29.319887 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:29.320609 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:29.320809 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:29.320884 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:29.328360 (+ 7476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:59223: BlockingRecv error: recv got EOF from 127.0.191.194:59223 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 26 06:53:29 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:29 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:29.436106 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:29.431413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:43069)
0926 06:53:29.431595 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:29.431606 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:29.431688 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:29.431815 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:29.431815 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:29.431996 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:29.432039 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:29.432041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:29.432707 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:29.432708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:29.433226 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:29.433227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:29.433663 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:29.433664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:29.433732 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:29.434177 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:29.434181 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:29.436012 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":95,"threads_started":1}
W20260926 06:53:29.436156 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:29.431501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:43069 (local address 127.0.191.254:35967)
0926 06:53:29.431577 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:29.431580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:29.431671 (+ 91us) server_negotiation.cc:407] Connection header received
0926 06:53:29.431742 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:29.431743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:29.431757 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:29.431810 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:29.432122 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:29.432624 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:29.433306 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:29.433582 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:29.433663 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:29.436091 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:43069: BlockingRecv error: recv got EOF from 127.0.191.195:43069 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 26 06:53:30 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:30 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:30.207062 9956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:30.202769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:35823)
0926 06:53:30.202914 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:30.202927 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:30.202988 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:30.203143 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:30.203144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:30.203309 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:30.203356 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:30.203358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:30.203976 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:30.203976 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:30.204491 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:30.204492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:30.204803 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:30.204804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:30.204842 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:30.205281 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:30.205285 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:30.207003 (+ 1718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
W20260926 06:53:30.207206 9957 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:30.202824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:35823 (local address 127.0.191.254:35967)
0926 06:53:30.202965 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:30.202967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:30.202975 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:53:30.203056 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:30.203056 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:30.203066 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:30.203123 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:30.203418 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:30.203912 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:30.204557 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:30.204735 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:30.204790 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:30.207108 (+ 2318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:35823: BlockingRecv error: recv got EOF from 127.0.191.193:35823 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
Sep 26 06:53:30 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:30 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:30.333671 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:30.328872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:33603)
0926 06:53:30.329119 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:30.329132 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:30.329223 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:30.329444 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:30.329445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:30.329619 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:30.329665 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:30.329666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:30.330342 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:30.330343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:30.330881 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:30.330882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:30.331255 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:30.331256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:30.331317 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:30.331838 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:30.331842 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:30.333580 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":79,"threads_started":1}
W20260926 06:53:30.333719 9957 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:30.328954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:33603 (local address 127.0.191.254:35967)
0926 06:53:30.329123 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:30.329126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:30.329205 (+ 79us) server_negotiation.cc:407] Connection header received
0926 06:53:30.329331 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:30.329332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:30.329343 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:30.329432 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:30.329742 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:30.330259 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:30.330968 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:30.331166 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:30.331249 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:30.333656 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:33603: BlockingRecv error: recv got EOF from 127.0.191.194:33603 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
Sep 26 06:53:30 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:30 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:30.441069 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:30.436561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:60749)
0926 06:53:30.436681 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:30.436693 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:30.436787 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:30.436919 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:30.436920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:30.437094 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:30.437138 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:30.437140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:30.437860 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:30.437861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:30.438380 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:30.438382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:30.438755 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:30.438756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:30.438820 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:30.439317 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:30.439328 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:30.440976 (+ 1648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
W20260926 06:53:30.441104 9957 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:30.436685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:60749 (local address 127.0.191.254:35967)
0926 06:53:30.436752 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:30.436754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:30.436767 (+ 13us) server_negotiation.cc:407] Connection header received
0926 06:53:30.436852 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:30.436853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:30.436864 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:30.436904 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:30.437213 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:30.437779 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:30.438460 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:30.438664 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:30.438746 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:30.441045 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:60749: BlockingRecv error: recv got EOF from 127.0.191.195:60749 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 26 06:53:31 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:31 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:31.212141 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:31.207520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:40185)
0926 06:53:31.207652 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:31.207665 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:31.207745 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:31.207967 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:31.207967 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:31.208168 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:31.208212 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:31.208214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:31.208834 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:31.208834 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:31.209529 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:31.209531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:31.209853 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:31.209854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:31.209906 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:31.210382 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:31.210386 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:31.212071 (+ 1685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
W20260926 06:53:31.212271 9961 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:31.207552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:40185 (local address 127.0.191.254:35967)
0926 06:53:31.207675 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:31.207677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:31.207730 (+ 53us) server_negotiation.cc:407] Connection header received
0926 06:53:31.207868 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:31.207869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:31.207881 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:31.207943 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:31.208279 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:31.208769 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:31.209593 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:31.209784 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:31.209836 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:31.212158 (+ 2322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:40185: BlockingRecv error: recv got EOF from 127.0.191.193:40185 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
Sep 26 06:53:31 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:31 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:31.338779 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:31.334200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:52353)
0926 06:53:31.334338 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:31.334347 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:31.334401 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:31.334535 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:31.334535 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:31.334695 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:31.334738 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:31.334740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:31.335380 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:31.335381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:31.335886 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:31.335887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:31.336249 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:31.336250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:31.336316 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:31.336871 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:31.336876 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:31.338685 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
W20260926 06:53:31.338835 9961 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:31.334283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:52353 (local address 127.0.191.254:35967)
0926 06:53:31.334347 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:31.334349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:31.334388 (+ 39us) server_negotiation.cc:407] Connection header received
0926 06:53:31.334458 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:31.334459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:31.334467 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:31.334509 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:31.334793 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:31.335316 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:31.335944 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:31.336178 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:31.336228 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:31.338769 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:52353: BlockingRecv error: recv got EOF from 127.0.191.194:52353 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 26 06:53:31 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:31 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:31.446172 9963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:31.441530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:50287)
0926 06:53:31.441670 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:31.441683 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:31.441743 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:31.441889 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:31.441890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:31.442043 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:31.442088 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:31.442090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:31.442764 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:31.442765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:31.443312 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:31.443314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:31.443640 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:31.443641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:31.443680 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:31.444196 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:31.444202 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:31.446103 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
W20260926 06:53:31.446300 9961 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:31.441589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:50287 (local address 127.0.191.254:35967)
0926 06:53:31.441653 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:31.441656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:31.441742 (+ 86us) server_negotiation.cc:407] Connection header received
0926 06:53:31.441793 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:31.441793 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:31.441806 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:31.441874 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:31.442147 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:31.442704 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:31.443390 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:31.443579 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:31.443628 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:31.446191 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:50287: BlockingRecv error: recv got EOF from 127.0.191.195:50287 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 26 06:53:32 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:32 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:32.216995 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:32.212533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:59855)
0926 06:53:32.212652 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:32.212669 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:32.212717 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:32.212883 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:32.212883 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:32.213045 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:32.213091 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:32.213092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:32.213708 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:32.213708 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:32.214225 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:32.214226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:32.214555 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:32.214556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:32.214593 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:32.215022 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:32.215027 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:32.216904 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
W20260926 06:53:32.217053 9965 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:32.212627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:59855 (local address 127.0.191.254:35967)
0926 06:53:32.212772 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:32.212774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:32.212780 (+ 6us) server_negotiation.cc:407] Connection header received
0926 06:53:32.212800 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:32.212800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:32.212810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:32.212868 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:32.213154 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:32.213653 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:32.214298 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:32.214494 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:32.214548 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:32.216979 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:59855: BlockingRecv error: recv got EOF from 127.0.191.193:59855 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
Sep 26 06:53:32 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:32 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:32.343688 9966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:32.339459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:47795)
0926 06:53:32.339566 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:32.339576 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:32.339642 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:32.339760 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:32.339760 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:32.339932 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:32.339974 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:32.339975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:32.340647 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:32.340648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:32.341151 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:32.341152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:32.341502 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:32.341503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:32.341547 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:32.341933 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:32.341937 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:32.343622 (+ 1685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":60,"threads_started":1}
W20260926 06:53:32.343793 9965 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:32.339515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:47795 (local address 127.0.191.254:35967)
0926 06:53:32.339581 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:32.339583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:32.339634 (+ 51us) server_negotiation.cc:407] Connection header received
0926 06:53:32.339698 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:32.339698 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:32.339708 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:32.339746 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:32.340029 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:32.340583 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:32.341214 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:32.341437 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:32.341483 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:32.343706 (+ 2223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:47795: BlockingRecv error: recv got EOF from 127.0.191.194:47795 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 06:53:32 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:32 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:32.450814 9967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:32.446634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:60501)
0926 06:53:32.446803 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:32.446814 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:32.446872 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:32.446990 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:32.446991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:32.447147 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:32.447194 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:32.447195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:32.447793 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:32.447794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:32.448302 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:32.448303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:32.448590 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:32.448590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:32.448628 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:32.449044 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:32.449048 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:32.450760 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":108,"threads_started":1}
W20260926 06:53:32.450932 9965 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:32.446674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:60501 (local address 127.0.191.254:35967)
0926 06:53:32.446736 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:32.446738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:32.446871 (+ 133us) server_negotiation.cc:407] Connection header received
0926 06:53:32.446912 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:32.446913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:32.446925 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:32.446972 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:32.447248 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:32.447724 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:32.448363 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:32.448533 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:32.448579 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:32.450837 (+ 2258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:60501: BlockingRecv error: recv got EOF from 127.0.191.195:60501 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 26 06:53:33 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:33 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:33.221781 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:33.217489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:37839)
0926 06:53:33.217711 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:33.217726 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:33.217784 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:33.217918 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:33.217919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:33.218080 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:33.218126 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:33.218128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:33.218740 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:33.218740 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:33.219250 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:33.219251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:33.219558 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:33.219558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:33.219597 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:33.220049 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:33.220053 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:33.221721 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":172,"threads_started":1}
W20260926 06:53:33.221946 9969 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:33.217565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:37839 (local address 127.0.191.254:35967)
0926 06:53:33.217699 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:33.217701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:33.217785 (+ 84us) server_negotiation.cc:407] Connection header received
0926 06:53:33.217839 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:33.217840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:33.217850 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:33.217900 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:33.218187 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:33.218676 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:33.219315 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:33.219498 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:33.219546 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:33.221818 (+ 2272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:37839: BlockingRecv error: recv got EOF from 127.0.191.193:37839 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
Sep 26 06:53:33 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:33 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:33.348659 9970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:33.344135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:46465)
0926 06:53:33.344269 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:33.344280 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:33.344339 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:33.344495 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:33.344497 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:33.344681 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:33.344740 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:33.344742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:33.345411 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:33.345412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:33.345943 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:33.345944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:33.346311 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:33.346312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:33.346395 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:33.346867 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:33.346872 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:33.348564 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
W20260926 06:53:33.348701 9969 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:33.344230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:46465 (local address 127.0.191.254:35967)
0926 06:53:33.344309 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:33.344311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:33.344322 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:53:33.344403 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:33.344404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:33.344415 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:33.344490 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:33.344829 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:33.345311 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:33.346027 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:33.346225 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:33.346302 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:33.348639 (+ 2337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:46465: BlockingRecv error: recv got EOF from 127.0.191.194:46465 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 26 06:53:33 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:33 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:33.455777 9971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:33.451327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:48107)
0926 06:53:33.451481 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:33.451492 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:33.451562 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:33.451685 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:33.451686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:33.451852 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:33.451898 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:33.451899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:33.452570 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:33.452571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:33.453081 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:33.453082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:33.453484 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:33.453484 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:33.453548 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:33.453971 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:33.453976 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:33.455714 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
W20260926 06:53:33.455937 9969 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:33.451409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:48107 (local address 127.0.191.254:35967)
0926 06:53:33.451496 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:33.451498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:33.451570 (+ 72us) server_negotiation.cc:407] Connection header received
0926 06:53:33.451599 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:33.451600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:33.451612 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:33.451681 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:33.452001 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:33.452495 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:33.453171 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:33.453358 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:33.453483 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:33.455838 (+ 2355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:48107: BlockingRecv error: recv got EOF from 127.0.191.195:48107 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 26 06:53:34 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:34 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:34.226881 9972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:34.222305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:33615)
0926 06:53:34.222453 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:34.222464 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:34.222525 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:34.222657 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:34.222658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:34.222819 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:34.222864 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:34.222866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:34.223513 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:34.223514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:34.224195 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:34.224197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:34.224603 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:34.224603 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:34.224669 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:34.225096 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:34.225101 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:34.226825 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
W20260926 06:53:34.227020 9973 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:34.222383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:33615 (local address 127.0.191.254:35967)
0926 06:53:34.222487 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:34.222489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:34.222511 (+ 22us) server_negotiation.cc:407] Connection header received
0926 06:53:34.222579 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:34.222579 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:34.222589 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:34.222638 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:34.222941 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:34.223449 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:34.224300 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:34.224497 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:34.224601 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:34.226933 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:33615: BlockingRecv error: recv got EOF from 127.0.191.193:33615 (error 108)
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":47,"threads_started":1}
Sep 26 06:53:34 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:34 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:34.360146 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:34.349365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:59449)
0926 06:53:34.349557 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:34.349571 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:34.349656 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:34.349827 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:34.349828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:34.350010 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:34.350053 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:34.350055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:34.350701 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:34.350703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:34.351231 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:34.351232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:34.351617 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:34.351618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:34.351693 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:34.357978 (+ 6285us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:34.357983 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:34.360049 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":69,"threads_started":1}
W20260926 06:53:34.360199 9973 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:34.349448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:59449 (local address 127.0.191.254:35967)
0926 06:53:34.349534 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:34.349536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:34.349636 (+ 100us) server_negotiation.cc:407] Connection header received
0926 06:53:34.349723 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:34.349724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:34.349736 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:34.349819 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:34.350138 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:34.350623 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:34.351322 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:34.351518 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:34.351619 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:34.360136 (+ 8517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:59449: BlockingRecv error: recv got EOF from 127.0.191.194:59449 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 06:53:34 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:34 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.195@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:34.461194 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:34.456300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.195:51289)
0926 06:53:34.456488 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:34.456499 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:34.456569 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:34.456728 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:34.456729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:34.456918 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:34.456960 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:34.456962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:34.457717 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:34.457718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:34.458266 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:34.458267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:34.458643 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:34.458643 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:34.458708 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:34.459122 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:34.459126 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:34.461088 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":108,"threads_started":1}
W20260926 06:53:34.461227 9973 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:34.456383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:51289 (local address 127.0.191.254:35967)
0926 06:53:34.456482 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:34.456484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:34.456547 (+ 63us) server_negotiation.cc:407] Connection header received
0926 06:53:34.456628 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:34.456629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:34.456644 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:34.456718 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:34.457041 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:34.457641 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:34.458350 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:34.458576 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:34.458643 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:34.461162 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.195:51289: BlockingRecv error: recv got EOF from 127.0.191.195:51289 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 26 06:53:35 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:35 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.193@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:35.231987 9976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:35.227367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.193:52321)
0926 06:53:35.227517 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:35.227555 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:35.227613 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:35.227805 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:35.227806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:35.227967 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:35.228026 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:35.228028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.228660 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.228661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:35.229218 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:35.229219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.229598 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.229599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:35.229643 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:35.230046 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:35.230050 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:35.231931 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
W20260926 06:53:35.232126 9977 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:35.227465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:52321 (local address 127.0.191.254:35967)
0926 06:53:35.227588 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:35.227590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:35.227598 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:53:35.227677 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:35.227678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:35.227695 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:35.227762 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:35.228100 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.228584 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.229290 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.229532 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.229584 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:35.232037 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.193:52321: BlockingRecv error: recv got EOF from 127.0.191.193:52321 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":67,"threads_started":1}
Sep 26 06:53:35 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:53:35 dist-test-slave-ws74 krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.191.194@KRBTEST.COM for oryx/127.0.191.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:53:35.365763 9978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:35.360682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:35967 (local address 127.0.191.194:35081)
0926 06:53:35.360832 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:35.360842 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:35.360900 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:35.361088 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:35.361089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:35.361250 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:35.361299 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:35.361300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.362037 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.362038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:35.362595 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:35.362596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.362962 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.362962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:35.363006 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:35.363561 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:35.363566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:35.365699 (+ 2133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.191.254:35967: Server oryx/127.0.191.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
W20260926 06:53:35.365926 9977 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:35.360736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:35081 (local address 127.0.191.254:35967)
0926 06:53:35.360826 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:35.360828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:35.360898 (+ 70us) server_negotiation.cc:407] Connection header received
0926 06:53:35.360945 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:35.360946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:35.360956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:35.361007 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:35.361366 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.361949 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.362695 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.362887 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.362954 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:35.365853 (+ 2899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.191.194:35081: BlockingRecv error: recv got EOF from 127.0.191.194:35081 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 06:53:35.387292 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 9500
I20260926 06:53:35.391580 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 9652
I20260926 06:53:35.395788 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 9785
I20260926 06:53:35.399824 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 9430
2026-09-26T06:53:35Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17378 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T06:53:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:35Z Disabled control of system clock
I20260926 06:53:35.426522 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:43833
--webserver_interface=127.0.191.254
--webserver_port=0
--builtin_ntp_servers=127.0.191.212:35723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:43833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 06:53:35.505270 9982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:35.505496 9982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:35.505517 9982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:35.506673 9982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:35.506711 9982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:35.506723 9982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:35.506736 9982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:35.506747 9982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:35.508001 9982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:35723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:43833
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:43833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:35.508184 9982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:35.508405 9982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:35.513305 9987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:35.513345 9982 server_base.cc:1034] running on GCE node
W20260926 06:53:35.513477 9990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:35.513366 9988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:35.513810 9982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:35.514091 9982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:35.515244 9982 hybrid_clock.cc:648] HybridClock initialized: now 1790405615515228 us; error 39 us; skew 500 ppm
I20260926 06:53:35.516455 9982 webserver.cc:460] Webserver started at http://127.0.191.254:33671/ using document root <none> and password file <none>
I20260926 06:53:35.516654 9982 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:35.516693 9982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:35.516778 9982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:35.517840 9982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "e872b21de06c429bba5c85d04856cf4a"
format_stamp: "Formatted at 2026-09-26 06:53:35 on dist-test-slave-ws74"
server_key: "87c8a0d2b33cd22235c7b4306c1e9972"
server_key_iv: "081bb1faebb192b2be7138f9c7164c82"
server_key_version: "encryptionkey@0"
I20260926 06:53:35.518136 9982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "e872b21de06c429bba5c85d04856cf4a"
format_stamp: "Formatted at 2026-09-26 06:53:35 on dist-test-slave-ws74"
server_key: "87c8a0d2b33cd22235c7b4306c1e9972"
server_key_iv: "081bb1faebb192b2be7138f9c7164c82"
server_key_version: "encryptionkey@0"
I20260926 06:53:35.519424 9982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:35.520191 9996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:35.520469 9982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:35.520573 9982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "e872b21de06c429bba5c85d04856cf4a"
format_stamp: "Formatted at 2026-09-26 06:53:35 on dist-test-slave-ws74"
server_key: "87c8a0d2b33cd22235c7b4306c1e9972"
server_key_iv: "081bb1faebb192b2be7138f9c7164c82"
server_key_version: "encryptionkey@0"
I20260926 06:53:35.520650 9982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:35.540064 9982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:35.540370 9982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:35.540517 9982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:35.544399 9982 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:43833
I20260926 06:53:35.544431 10048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:43833 every 8 connection(s)
I20260926 06:53:35.544760 9982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:35.545576 10049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:35.548170 10049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a: Bootstrap starting.
I20260926 06:53:35.548749 10049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:35.548995 10049 log.cc:824] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:35.549679 10049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a: No bootstrap required, opened a new log
I20260926 06:53:35.550544 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 9982
I20260926 06:53:35.550647 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:35.550845 767 external_mini_cluster.cc:1428] Setting key ade28af89916f8081fed9e1a4634b358
I20260926 06:53:35.551630 10049 raft_consensus.cc:348] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e872b21de06c429bba5c85d04856cf4a" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 43833 } }
I20260926 06:53:35.551766 10049 raft_consensus.cc:374] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:35.551788 10049 raft_consensus.cc:729] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e872b21de06c429bba5c85d04856cf4a, State: Initialized, Role: FOLLOWER
I20260926 06:53:35.551951 10049 consensus_queue.cc:260] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e872b21de06c429bba5c85d04856cf4a" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 43833 } }
I20260926 06:53:35.552050 10049 raft_consensus.cc:388] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:35.552105 10049 raft_consensus.cc:482] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:35.552160 10049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:35.552668 10049 raft_consensus.cc:504] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e872b21de06c429bba5c85d04856cf4a" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 43833 } }
I20260926 06:53:35.552786 10049 leader_election.cc:302] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e872b21de06c429bba5c85d04856cf4a; no voters:
I20260926 06:53:35.552959 10049 leader_election.cc:288] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:35.553004 10054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:35.553144 10054 raft_consensus.cc:686] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [term 1 LEADER]: Becoming Leader. State: Replica: e872b21de06c429bba5c85d04856cf4a, State: Running, Role: LEADER
I20260926 06:53:35.553268 10049 sys_catalog.cc:564] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:35.553320 10054 consensus_queue.cc:237] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e872b21de06c429bba5c85d04856cf4a" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 43833 } }
I20260926 06:53:35.553761 10054 sys_catalog.cc:455] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e872b21de06c429bba5c85d04856cf4a. Latest consensus state: current_term: 1 leader_uuid: "e872b21de06c429bba5c85d04856cf4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e872b21de06c429bba5c85d04856cf4a" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 43833 } } }
I20260926 06:53:35.553767 10056 sys_catalog.cc:455] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e872b21de06c429bba5c85d04856cf4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e872b21de06c429bba5c85d04856cf4a" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 43833 } } }
I20260926 06:53:35.553957 10056 sys_catalog.cc:458] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:35.554147 10054 sys_catalog.cc:458] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:35.554433 10063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:35.554983 10063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:35.556648 10063 catalog_manager.cc:1347] Generated new cluster ID: e3a8b612432b41ac94617272df00f3fd
I20260926 06:53:35.556703 10063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:53:35.557155 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:35.551364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52184 (local address 127.0.191.254:43833)
0926 06:53:35.551559 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:35.551568 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:35.551591 (+ 23us) server_negotiation.cc:407] Connection header received
0926 06:53:35.551691 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:35.551696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:35.551756 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:35.551939 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:35.552177 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.553720 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.554933 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.555576 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.555687 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:35.556467 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:35.556471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:35.556471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:35.556708 (+ 237us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:35.556753 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:35.556769 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:35.556829 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:35.556904 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 06:53:35.556980 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260926 06:53:35.563683 10063 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:35.564354 10063 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:35.575946 10063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e872b21de06c429bba5c85d04856cf4a: Generated new TSK 0
I20260926 06:53:35.576146 10063 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:53:35.581861 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:43833
--builtin_ntp_servers=127.0.191.212:35723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 06:53:35.669415 10073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:35.669653 10073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:35.669689 10073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:35.670820 10073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:35.670897 10073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:35.670935 10073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:53:35.672816 10073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:35723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:43833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:35.673075 10073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:35.673414 10073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:35.678992 10078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:35.679039 10079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:35.679045 10081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:35.679370 10073 server_base.cc:1034] running on GCE node
I20260926 06:53:35.679603 10073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:35.679864 10073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:35.681068 10073 hybrid_clock.cc:648] HybridClock initialized: now 1790405615681049 us; error 34 us; skew 500 ppm
I20260926 06:53:35.682555 10073 webserver.cc:460] Webserver started at http://127.0.191.193:35997/ using document root <none> and password file <none>
I20260926 06:53:35.682825 10073 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:35.682904 10073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:35.683079 10073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:35.684247 10073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "6f2ba24200a2404f9ecd5ef4224c563a"
format_stamp: "Formatted at 2026-09-26 06:53:35 on dist-test-slave-ws74"
server_key: "f393631ba2d6630a6a01800bb17e1fae"
server_key_iv: "804f1129dbef7e89e4bc6d68530f8532"
server_key_version: "encryptionkey@0"
I20260926 06:53:35.684628 10073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "6f2ba24200a2404f9ecd5ef4224c563a"
format_stamp: "Formatted at 2026-09-26 06:53:35 on dist-test-slave-ws74"
server_key: "f393631ba2d6630a6a01800bb17e1fae"
server_key_iv: "804f1129dbef7e89e4bc6d68530f8532"
server_key_version: "encryptionkey@0"
I20260926 06:53:35.686120 10073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:53:35.686964 10087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:35.687135 10073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:35.687249 10073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "6f2ba24200a2404f9ecd5ef4224c563a"
format_stamp: "Formatted at 2026-09-26 06:53:35 on dist-test-slave-ws74"
server_key: "f393631ba2d6630a6a01800bb17e1fae"
server_key_iv: "804f1129dbef7e89e4bc6d68530f8532"
server_key_version: "encryptionkey@0"
I20260926 06:53:35.687325 10073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:35.705476 10073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:35.705798 10073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:35.705957 10073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:35.706189 10073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:35.706516 10073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:35.706550 10073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:35.706573 10073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:35.706621 10073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:35.712538 10073 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:37161
I20260926 06:53:35.712582 10200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:37161 every 8 connection(s)
I20260926 06:53:35.712935 10073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:53:35.717474 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 10073
I20260926 06:53:35.717577 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:53:35.717849 767 external_mini_cluster.cc:1428] Setting key d9b9493188fc4920402baa219b543584
I20260926 06:53:35.718633 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:35.713747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:47273 (local address 127.0.191.254:43833)
0926 06:53:35.713802 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:35.713804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:35.714066 (+ 262us) server_negotiation.cc:407] Connection header received
0926 06:53:35.714547 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:35.714548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:35.714560 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:35.714623 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:35.715315 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.715812 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.716868 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.717058 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.717116 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:35.718119 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:35.718121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:35.718121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:35.718126 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:35.718174 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:35.718179 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:35.718227 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:35.718527 (+ 300us) server_negotiation.cc:299] Negotiation successful
0926 06:53:35.718558 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:53:35.718871 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:43833
--builtin_ntp_servers=127.0.191.212:35723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 06:53:35.718914 10203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:35.713749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:43833 (local address 127.0.191.193:47273)
0926 06:53:35.713945 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:35.713979 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:35.714468 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:35.714698 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:35.714702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:35.714711 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:35.715214 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:35.715218 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.715926 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.715927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:35.716782 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:35.716784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.717176 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.717177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:35.717342 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:35.717955 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:35.717955 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:35.717995 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:35.718032 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:35.718436 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:35.718442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:35.718445 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:35.718517 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:53:35.718645 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260926 06:53:35.719570 10201 heartbeater.cc:347] Connected to a master server at 127.0.191.254:43833
I20260926 06:53:35.719673 10201 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:35.719857 10201 heartbeater.cc:511] Master 127.0.191.254:43833 requested a full tablet report, sending...
I20260926 06:53:35.720386 10013 ts_manager.cc:194] Registered new tserver with Master: 6f2ba24200a2404f9ecd5ef4224c563a (127.0.191.193:37161)
I20260926 06:53:35.720978 10013 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.191.193:47273
W20260926 06:53:35.798624 10204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:35.798828 10204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:35.798849 10204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:35.800014 10204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:35.800055 10204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:35.800078 10204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:53:35.801409 10204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:35723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:43833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:35.801652 10204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:35.801939 10204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:35.806833 10210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:35.806870 10204 server_base.cc:1034] running on GCE node
W20260926 06:53:35.806842 10212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:35.806877 10209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:35.807276 10204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:35.807471 10204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:35.808666 10204 hybrid_clock.cc:648] HybridClock initialized: now 1790405615808652 us; error 32 us; skew 500 ppm
I20260926 06:53:35.809963 10204 webserver.cc:460] Webserver started at http://127.0.191.194:38201/ using document root <none> and password file <none>
I20260926 06:53:35.810225 10204 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:35.810272 10204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:35.810438 10204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:35.811509 10204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "3873abd0a8cb42e68bb177afc9ad7e98"
format_stamp: "Formatted at 2026-09-26 06:53:35 on dist-test-slave-ws74"
server_key: "8f0e372994327c5d50b2162bdd941059"
server_key_iv: "617867164f370c93ed096248be0b1f1d"
server_key_version: "encryptionkey@0"
I20260926 06:53:35.811862 10204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "3873abd0a8cb42e68bb177afc9ad7e98"
format_stamp: "Formatted at 2026-09-26 06:53:35 on dist-test-slave-ws74"
server_key: "8f0e372994327c5d50b2162bdd941059"
server_key_iv: "617867164f370c93ed096248be0b1f1d"
server_key_version: "encryptionkey@0"
I20260926 06:53:35.813207 10204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:53:35.813977 10218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:35.814170 10204 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:35.814260 10204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "3873abd0a8cb42e68bb177afc9ad7e98"
format_stamp: "Formatted at 2026-09-26 06:53:35 on dist-test-slave-ws74"
server_key: "8f0e372994327c5d50b2162bdd941059"
server_key_iv: "617867164f370c93ed096248be0b1f1d"
server_key_version: "encryptionkey@0"
I20260926 06:53:35.814329 10204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:35.829200 10204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:35.829583 10204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:35.829720 10204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:35.829927 10204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:35.830262 10204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:35.830294 10204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:35.830314 10204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:35.830327 10204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:35.835932 10204 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:35871
I20260926 06:53:35.835976 10331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:35871 every 8 connection(s)
I20260926 06:53:35.836303 10204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:53:35.841223 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:35.836991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:54131 (local address 127.0.191.254:43833)
0926 06:53:35.837037 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:35.837039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:35.837285 (+ 246us) server_negotiation.cc:407] Connection header received
0926 06:53:35.837825 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:35.837826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:35.837835 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:35.837900 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:35.838546 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.839021 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.839950 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.840126 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.840198 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:35.841007 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:35.841008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:35.841008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:35.841012 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:35.841022 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:35.841024 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:35.841061 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:35.841149 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 06:53:35.841168 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:53:35.841434 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:35.836949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:43833 (local address 127.0.191.194:54131)
0926 06:53:35.837156 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:35.837195 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:35.837738 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:35.837957 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:35.837961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:35.837969 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:35.838448 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:35.838453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.839096 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.839096 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:35.839878 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:35.839880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.840218 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.840218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:35.840324 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:35.840893 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:35.840893 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:35.840920 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:35.840940 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:35.841074 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:35.841077 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:35.841079 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:53:35.841129 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 06:53:35.841200 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260926 06:53:35.841971 10332 heartbeater.cc:347] Connected to a master server at 127.0.191.254:43833
I20260926 06:53:35.842056 10332 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:35.842211 10332 heartbeater.cc:511] Master 127.0.191.254:43833 requested a full tablet report, sending...
I20260926 06:53:35.842543 10013 ts_manager.cc:194] Registered new tserver with Master: 3873abd0a8cb42e68bb177afc9ad7e98 (127.0.191.194:35871)
I20260926 06:53:35.842862 10013 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.191.194:54131
I20260926 06:53:35.843788 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 10204
I20260926 06:53:35.843863 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:53:35.843992 767 external_mini_cluster.cc:1428] Setting key a5241d03be1856777a983c01f7be3a73
I20260926 06:53:35.844794 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:43833
--builtin_ntp_servers=127.0.191.212:35723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 06:53:35.923638 10335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:35.923852 10335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:35.923887 10335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:35.924954 10335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:35.924986 10335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:35.925009 10335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:53:35.926266 10335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:35723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:43833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:35.926461 10335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:35.926690 10335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:35.931416 10343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:35.931427 10341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:35.931653 10335 server_base.cc:1034] running on GCE node
W20260926 06:53:35.931689 10340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:35.931972 10335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:35.932199 10335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:35.933336 10335 hybrid_clock.cc:648] HybridClock initialized: now 1790405615933324 us; error 26 us; skew 500 ppm
I20260926 06:53:35.934489 10335 webserver.cc:460] Webserver started at http://127.0.191.195:42347/ using document root <none> and password file <none>
I20260926 06:53:35.934736 10335 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:35.934803 10335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:35.934933 10335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:35.935982 10335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "1b0259e547724f53a62a930f07e8b0d9"
format_stamp: "Formatted at 2026-09-26 06:53:35 on dist-test-slave-ws74"
server_key: "d96854a7979733ac02872e466e6f57ca"
server_key_iv: "4ba1d67acdf582fd16610d383d5c0ace"
server_key_version: "encryptionkey@0"
I20260926 06:53:35.936334 10335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "1b0259e547724f53a62a930f07e8b0d9"
format_stamp: "Formatted at 2026-09-26 06:53:35 on dist-test-slave-ws74"
server_key: "d96854a7979733ac02872e466e6f57ca"
server_key_iv: "4ba1d67acdf582fd16610d383d5c0ace"
server_key_version: "encryptionkey@0"
I20260926 06:53:35.937738 10335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:53:35.938539 10349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:35.938730 10335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:53:35.938823 10335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "1b0259e547724f53a62a930f07e8b0d9"
format_stamp: "Formatted at 2026-09-26 06:53:35 on dist-test-slave-ws74"
server_key: "d96854a7979733ac02872e466e6f57ca"
server_key_iv: "4ba1d67acdf582fd16610d383d5c0ace"
server_key_version: "encryptionkey@0"
I20260926 06:53:35.938902 10335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:35.951732 10335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:35.952009 10335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:35.952162 10335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:35.952378 10335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:35.952709 10335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:35.952754 10335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:35.952790 10335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:35.952819 10335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:35.958679 10335 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:38963
I20260926 06:53:35.958719 10462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:38963 every 8 connection(s)
I20260926 06:53:35.959016 10335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:53:35.959262 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 10335
I20260926 06:53:35.959355 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:53:35.959563 767 external_mini_cluster.cc:1428] Setting key f3427e8dbdbd198628ad046c44457de0
I20260926 06:53:35.964411 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:35.959772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:48815 (local address 127.0.191.254:43833)
0926 06:53:35.959838 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:35.959840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:35.960111 (+ 271us) server_negotiation.cc:407] Connection header received
0926 06:53:35.960840 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:35.960840 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:35.960849 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:35.960901 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:35.961569 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.962062 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.963094 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.963286 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.963338 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:35.964189 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:35.964191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:35.964191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:35.964195 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:35.964208 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:35.964212 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:35.964254 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:35.964332 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 06:53:35.964354 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:53:35.964632 10465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:35.959789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:43833 (local address 127.0.191.195:48815)
0926 06:53:35.959985 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:35.960027 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:35.960741 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:35.960958 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:35.960961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:35.960970 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:35.961498 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:35.961503 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.962173 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.962174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:35.962989 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:35.962991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:35.963408 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:35.963409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:35.963570 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:35.964045 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:35.964046 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:35.964073 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:35.964112 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:35.964267 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:35.964271 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:35.964273 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:53:35.964323 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 06:53:35.964399 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260926 06:53:35.965091 10463 heartbeater.cc:347] Connected to a master server at 127.0.191.254:43833
I20260926 06:53:35.965185 10463 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:35.965365 10463 heartbeater.cc:511] Master 127.0.191.254:43833 requested a full tablet report, sending...
I20260926 06:53:35.965801 10013 ts_manager.cc:194] Registered new tserver with Master: 1b0259e547724f53a62a930f07e8b0d9 (127.0.191.195:38963)
I20260926 06:53:35.966120 10013 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.191.195:48815
I20260926 06:53:35.971342 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 06:53:35.972990 10053 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:35.972597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52192 (local address 127.0.191.254:43833)
0926 06:53:35.972647 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:35.972649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:35.972700 (+ 51us) server_negotiation.cc:407] Connection header received
0926 06:53:35.972750 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:35.972751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:35.972760 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:35.972802 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:35.972910 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52192: BlockingRecv error: recv got EOF from 127.0.0.1:52192 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:53:35.973182 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 10073
I20260926 06:53:35.977804 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 10204
I20260926 06:53:35.981882 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 10335
I20260926 06:53:35.985837 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 9982
2026-09-26T06:53:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (585 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T06:53:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:36Z Disabled control of system clock
I20260926 06:53:36.011696 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33487
--webserver_interface=127.0.191.254
--webserver_port=0
--builtin_ntp_servers=127.0.191.212:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:33487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 06:53:36.094035 10474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:36.094267 10474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:36.094286 10474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:36.095362 10474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:36.095427 10474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:36.095453 10474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:36.095474 10474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:36.095503 10474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:36.096824 10474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:33487
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:36.097013 10474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:36.097244 10474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:53:36.102054 10474 server_base.cc:1034] running on GCE node
W20260926 06:53:36.102023 10480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:36.102027 10482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:36.102072 10479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:36.102435 10474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:36.102757 10474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:36.103935 10474 hybrid_clock.cc:648] HybridClock initialized: now 1790405616103911 us; error 35 us; skew 500 ppm
I20260926 06:53:36.105325 10474 webserver.cc:460] Webserver started at http://127.0.191.254:46289/ using document root <none> and password file <none>
I20260926 06:53:36.105592 10474 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:36.105665 10474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:36.105813 10474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:36.106863 10474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "a02f0844925b407282a8cf44d000522e"
format_stamp: "Formatted at 2026-09-26 06:53:36 on dist-test-slave-ws74"
server_key: "0e71d5d55836a8ef3e2c9f2d51ce7fde"
server_key_iv: "c6c778555d8c677469b43d09b121b13f"
server_key_version: "encryptionkey@0"
I20260926 06:53:36.107256 10474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "a02f0844925b407282a8cf44d000522e"
format_stamp: "Formatted at 2026-09-26 06:53:36 on dist-test-slave-ws74"
server_key: "0e71d5d55836a8ef3e2c9f2d51ce7fde"
server_key_iv: "c6c778555d8c677469b43d09b121b13f"
server_key_version: "encryptionkey@0"
I20260926 06:53:36.108541 10474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:53:36.109256 10488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:36.109470 10474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:36.109591 10474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "a02f0844925b407282a8cf44d000522e"
format_stamp: "Formatted at 2026-09-26 06:53:36 on dist-test-slave-ws74"
server_key: "0e71d5d55836a8ef3e2c9f2d51ce7fde"
server_key_iv: "c6c778555d8c677469b43d09b121b13f"
server_key_version: "encryptionkey@0"
I20260926 06:53:36.109665 10474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:36.112824 10474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:36.113087 10474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:36.113235 10474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:36.116807 10474 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:33487
I20260926 06:53:36.116873 10540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:33487 every 8 connection(s)
I20260926 06:53:36.117204 10474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:36.117919 10541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:36.120568 10541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e: Bootstrap starting.
I20260926 06:53:36.121134 10541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:36.121487 10541 log.cc:824] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:36.122193 10541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e: No bootstrap required, opened a new log
I20260926 06:53:36.123745 10541 raft_consensus.cc:348] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02f0844925b407282a8cf44d000522e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33487 } }
I20260926 06:53:36.123881 10541 raft_consensus.cc:374] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:36.123903 10541 raft_consensus.cc:729] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02f0844925b407282a8cf44d000522e, State: Initialized, Role: FOLLOWER
I20260926 06:53:36.124053 10541 consensus_queue.cc:260] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02f0844925b407282a8cf44d000522e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33487 } }
I20260926 06:53:36.124132 10541 raft_consensus.cc:388] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:36.124154 10541 raft_consensus.cc:482] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:36.124217 10541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:36.124728 10541 raft_consensus.cc:504] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02f0844925b407282a8cf44d000522e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33487 } }
I20260926 06:53:36.124809 10541 leader_election.cc:302] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a02f0844925b407282a8cf44d000522e; no voters:
I20260926 06:53:36.124998 10541 leader_election.cc:288] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:36.125048 10544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:36.125260 10544 raft_consensus.cc:686] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [term 1 LEADER]: Becoming Leader. State: Replica: a02f0844925b407282a8cf44d000522e, State: Running, Role: LEADER
I20260926 06:53:36.125294 10541 sys_catalog.cc:564] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:36.125398 10544 consensus_queue.cc:237] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02f0844925b407282a8cf44d000522e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33487 } }
I20260926 06:53:36.125793 10545 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a02f0844925b407282a8cf44d000522e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02f0844925b407282a8cf44d000522e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33487 } } }
I20260926 06:53:36.125873 10545 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:36.125984 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 10474
I20260926 06:53:36.126096 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:36.126284 767 external_mini_cluster.cc:1428] Setting key 245bffff721c82c51406b5077be455f4
I20260926 06:53:36.126634 10546 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a02f0844925b407282a8cf44d000522e. Latest consensus state: current_term: 1 leader_uuid: "a02f0844925b407282a8cf44d000522e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02f0844925b407282a8cf44d000522e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33487 } } }
I20260926 06:53:36.126746 10546 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:36.127702 10550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:36.128201 10550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:36.129082 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:36.126983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33040 (local address 127.0.191.254:33487)
0926 06:53:36.127162 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:36.127170 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:36.127193 (+ 23us) server_negotiation.cc:407] Connection header received
0926 06:53:36.127280 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:36.127286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:36.127317 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:36.127453 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:36.128491 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:36.128494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:36.128495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:36.128498 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:36.128536 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:36.128550 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:36.128620 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:36.128730 (+ 110us) server_negotiation.cc:299] Negotiation successful
0926 06:53:36.128898 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
I20260926 06:53:36.129983 10550 catalog_manager.cc:1347] Generated new cluster ID: 4a91302306f04b8a920cb734d93ab1b8
I20260926 06:53:36.130040 10550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:53:36.137355 10550 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:36.137547 10550 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:36.142839 10550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a02f0844925b407282a8cf44d000522e: Generated new TSK 0
I20260926 06:53:36.143009 10550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:53:36.149735 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33487
--builtin_ntp_servers=127.0.191.212:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 06:53:36.230259 10565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:36.230491 10565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:36.230531 10565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:36.231698 10565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:36.231761 10565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:36.231806 10565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:53:36.233143 10565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:36.233459 10565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:36.233760 10565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:36.238945 10573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:36.239089 10571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:36.238935 10570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:36.238931 10565 server_base.cc:1034] running on GCE node
I20260926 06:53:36.239420 10565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:36.239696 10565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:36.240875 10565 hybrid_clock.cc:648] HybridClock initialized: now 1790405616240862 us; error 29 us; skew 500 ppm
I20260926 06:53:36.242278 10565 webserver.cc:460] Webserver started at http://127.0.191.193:44601/ using document root <none> and password file <none>
I20260926 06:53:36.242547 10565 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:36.242600 10565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:36.242717 10565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:36.244107 10565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "4bf68bfd6795438e8b218e88fae2a6cf"
format_stamp: "Formatted at 2026-09-26 06:53:36 on dist-test-slave-ws74"
server_key: "b26f28c9679616c99bc42910fc9baf6a"
server_key_iv: "9d97bb7d5e3fe1c0453f6a796b89812a"
server_key_version: "encryptionkey@0"
I20260926 06:53:36.244506 10565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "4bf68bfd6795438e8b218e88fae2a6cf"
format_stamp: "Formatted at 2026-09-26 06:53:36 on dist-test-slave-ws74"
server_key: "b26f28c9679616c99bc42910fc9baf6a"
server_key_iv: "9d97bb7d5e3fe1c0453f6a796b89812a"
server_key_version: "encryptionkey@0"
I20260926 06:53:36.246129 10565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260926 06:53:36.246989 10579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:36.247197 10565 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:53:36.247288 10565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "4bf68bfd6795438e8b218e88fae2a6cf"
format_stamp: "Formatted at 2026-09-26 06:53:36 on dist-test-slave-ws74"
server_key: "b26f28c9679616c99bc42910fc9baf6a"
server_key_iv: "9d97bb7d5e3fe1c0453f6a796b89812a"
server_key_version: "encryptionkey@0"
I20260926 06:53:36.247351 10565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:36.252393 10565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:36.252687 10565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:36.252830 10565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:36.253050 10565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:36.253443 10565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:36.253494 10565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:36.253521 10565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:36.253538 10565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:36.260123 10565 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:44583
I20260926 06:53:36.260200 10692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:44583 every 8 connection(s)
I20260926 06:53:36.260602 10565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:53:36.263814 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:36.261490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:42549 (local address 127.0.191.254:33487)
0926 06:53:36.261546 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:36.261549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:36.261837 (+ 288us) server_negotiation.cc:407] Connection header received
0926 06:53:36.262300 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:36.262301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:36.262307 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:36.262381 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:36.263562 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:36.263563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:36.263563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:36.263568 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:36.263585 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:36.263590 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:36.263660 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:36.263726 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 06:53:36.263741 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 06:53:36.264026 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:36.261495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33487 (local address 127.0.191.193:42549)
0926 06:53:36.261707 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:36.261749 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:36.262204 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:36.262434 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:36.262438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:36.262445 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:36.263409 (+ 964us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:36.263410 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:36.263438 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:36.263466 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:36.263658 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:36.263660 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:36.263663 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:36.263713 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 06:53:36.263759 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
I20260926 06:53:36.264479 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 10565
I20260926 06:53:36.264565 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:53:36.264542 10693 heartbeater.cc:347] Connected to a master server at 127.0.191.254:33487
I20260926 06:53:36.264626 10693 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:36.264736 10693 heartbeater.cc:511] Master 127.0.191.254:33487 requested a full tablet report, sending...
I20260926 06:53:36.264765 767 external_mini_cluster.cc:1428] Setting key 984502e34dbc3ce3b1ee033ad6b18540
I20260926 06:53:36.265200 10505 ts_manager.cc:194] Registered new tserver with Master: 4bf68bfd6795438e8b218e88fae2a6cf (127.0.191.193:44583)
I20260926 06:53:36.265861 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33487
--builtin_ntp_servers=127.0.191.212:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 06:53:36.346055 10696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:36.346258 10696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:36.346279 10696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:36.347421 10696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:36.347460 10696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:36.347484 10696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:53:36.348879 10696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:36.349109 10696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:36.349406 10696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:36.354269 10702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:36.354287 10701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:36.354401 10696 server_base.cc:1034] running on GCE node
W20260926 06:53:36.354283 10704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:36.354679 10696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:36.354911 10696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:36.356073 10696 hybrid_clock.cc:648] HybridClock initialized: now 1790405616356059 us; error 33 us; skew 500 ppm
I20260926 06:53:36.357288 10696 webserver.cc:460] Webserver started at http://127.0.191.194:45809/ using document root <none> and password file <none>
I20260926 06:53:36.357542 10696 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:36.357612 10696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:36.357746 10696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:36.358793 10696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "9537af604fd84c80b52d3458a4ae0f7c"
format_stamp: "Formatted at 2026-09-26 06:53:36 on dist-test-slave-ws74"
server_key: "1f8fc6c484b53e2b66eac81390dd2992"
server_key_iv: "dc7a933cf63f41819ccf03908ac5aa37"
server_key_version: "encryptionkey@0"
I20260926 06:53:36.359159 10696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "9537af604fd84c80b52d3458a4ae0f7c"
format_stamp: "Formatted at 2026-09-26 06:53:36 on dist-test-slave-ws74"
server_key: "1f8fc6c484b53e2b66eac81390dd2992"
server_key_iv: "dc7a933cf63f41819ccf03908ac5aa37"
server_key_version: "encryptionkey@0"
I20260926 06:53:36.360498 10696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:53:36.361265 10710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:36.361455 10696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:36.361557 10696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "9537af604fd84c80b52d3458a4ae0f7c"
format_stamp: "Formatted at 2026-09-26 06:53:36 on dist-test-slave-ws74"
server_key: "1f8fc6c484b53e2b66eac81390dd2992"
server_key_iv: "dc7a933cf63f41819ccf03908ac5aa37"
server_key_version: "encryptionkey@0"
I20260926 06:53:36.361645 10696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:36.365000 10696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:36.365283 10696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:36.365446 10696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:36.365695 10696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:36.366055 10696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:36.366109 10696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:36.366150 10696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:36.366176 10696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:36.371898 10696 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:34679
I20260926 06:53:36.371942 10823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:34679 every 8 connection(s)
I20260926 06:53:36.372254 10696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:53:36.375018 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:36.372936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:55843 (local address 127.0.191.254:33487)
0926 06:53:36.372976 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:36.372978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:36.373215 (+ 237us) server_negotiation.cc:407] Connection header received
0926 06:53:36.373732 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:36.373733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:36.373738 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:36.373798 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:36.374841 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:36.374842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:36.374842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:36.374846 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:36.374861 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:36.374866 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:36.374913 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:36.374957 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 06:53:36.374968 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:53:36.375265 10826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:36.372905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33487 (local address 127.0.191.194:55843)
0926 06:53:36.373092 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:36.373138 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:36.373646 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:36.373849 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:36.373852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:36.373859 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:36.374712 (+ 853us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:36.374712 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:36.374738 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:36.374762 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:36.374909 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:36.374911 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:36.374913 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:53:36.374962 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 06:53:36.375014 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260926 06:53:36.375738 10824 heartbeater.cc:347] Connected to a master server at 127.0.191.254:33487
I20260926 06:53:36.375836 10824 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:36.375976 10824 heartbeater.cc:511] Master 127.0.191.254:33487 requested a full tablet report, sending...
I20260926 06:53:36.376317 10505 ts_manager.cc:194] Registered new tserver with Master: 9537af604fd84c80b52d3458a4ae0f7c (127.0.191.194:34679)
I20260926 06:53:36.380162 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 10696
I20260926 06:53:36.380282 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:53:36.380497 767 external_mini_cluster.cc:1428] Setting key 35a5eceeae9f14014cc0e239baf703b8
I20260926 06:53:36.381430 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33487
--builtin_ntp_servers=127.0.191.212:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 06:53:36.461067 10827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:36.461269 10827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:36.461293 10827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:36.462491 10827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:36.462532 10827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:36.462587 10827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:53:36.463927 10827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:36.464171 10827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:36.464463 10827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:36.469508 10835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:36.469569 10827 server_base.cc:1034] running on GCE node
W20260926 06:53:36.469519 10832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:36.469529 10833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:36.469892 10827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:36.470144 10827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:36.471310 10827 hybrid_clock.cc:648] HybridClock initialized: now 1790405616471298 us; error 27 us; skew 500 ppm
I20260926 06:53:36.472527 10827 webserver.cc:460] Webserver started at http://127.0.191.195:37015/ using document root <none> and password file <none>
I20260926 06:53:36.472765 10827 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:36.472834 10827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:36.472992 10827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:36.474116 10827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "d8d3b59f45b94a5aa46c8e3c8f183846"
format_stamp: "Formatted at 2026-09-26 06:53:36 on dist-test-slave-ws74"
server_key: "bc999cb8bf62e02fceceacb2859a251f"
server_key_iv: "9615b52f712366617f5ad720a6243b49"
server_key_version: "encryptionkey@0"
I20260926 06:53:36.474478 10827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "d8d3b59f45b94a5aa46c8e3c8f183846"
format_stamp: "Formatted at 2026-09-26 06:53:36 on dist-test-slave-ws74"
server_key: "bc999cb8bf62e02fceceacb2859a251f"
server_key_iv: "9615b52f712366617f5ad720a6243b49"
server_key_version: "encryptionkey@0"
I20260926 06:53:36.475744 10827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:53:36.476432 10841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:36.476588 10827 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:53:36.476683 10827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "d8d3b59f45b94a5aa46c8e3c8f183846"
format_stamp: "Formatted at 2026-09-26 06:53:36 on dist-test-slave-ws74"
server_key: "bc999cb8bf62e02fceceacb2859a251f"
server_key_iv: "9615b52f712366617f5ad720a6243b49"
server_key_version: "encryptionkey@0"
I20260926 06:53:36.476761 10827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:36.480068 10827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:36.480314 10827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:36.480455 10827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:36.480651 10827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:36.480931 10827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:36.480984 10827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:36.481021 10827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:36.481050 10827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:36.486928 10827 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:35205
I20260926 06:53:36.486969 10954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:35205 every 8 connection(s)
I20260926 06:53:36.487279 10827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:53:36.489984 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:36.487972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:42405 (local address 127.0.191.254:33487)
0926 06:53:36.488026 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:36.488028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:36.488233 (+ 205us) server_negotiation.cc:407] Connection header received
0926 06:53:36.488656 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:36.488657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:36.488663 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:36.488726 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:36.489803 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:36.489804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:36.489804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:53:36.489808 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:36.489821 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:53:36.489824 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:36.489869 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:36.489919 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 06:53:36.489933 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 06:53:36.490187 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:36.487952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33487 (local address 127.0.191.195:42405)
0926 06:53:36.488122 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:36.488160 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:36.488572 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:36.488771 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:36.488774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:36.488781 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:36.489672 (+ 891us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:53:36.489673 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:36.489698 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:53:36.489724 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:36.489872 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:36.489874 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:36.489877 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:53:36.489924 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 06:53:36.489967 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260926 06:53:36.490659 10955 heartbeater.cc:347] Connected to a master server at 127.0.191.254:33487
I20260926 06:53:36.490748 10955 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:36.490859 10955 heartbeater.cc:511] Master 127.0.191.254:33487 requested a full tablet report, sending...
I20260926 06:53:36.491189 10505 ts_manager.cc:194] Registered new tserver with Master: d8d3b59f45b94a5aa46c8e3c8f183846 (127.0.191.195:35205)
I20260926 06:53:36.495764 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 10827
I20260926 06:53:36.495877 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:53:36.496068 767 external_mini_cluster.cc:1428] Setting key 96b3b6929548ca05e4e48698afb00f35
I20260926 06:53:36.497077 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 06:53:36.498911 10556 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:53:36.498436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.0.191.254:33487)
0926 06:53:36.498492 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:36.498494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:36.498613 (+ 119us) server_negotiation.cc:407] Connection header received
0926 06:53:36.498687 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:36.498688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:36.498692 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:36.498736 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:36.498842 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33048: BlockingRecv error: recv got EOF from 127.0.0.1:33048 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:53:36.499179 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 10565
I20260926 06:53:36.504294 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 10696
I20260926 06:53:36.508358 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 10827
I20260926 06:53:36.512372 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 10474
2026-09-26T06:53:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (526 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:53:36 dist-test-slave-ws74 krb5kdc[10965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:53:36 dist-test-slave-ws74 krb5kdc[10965](info): set up 2 sockets
Sep 26 06:53:36 dist-test-slave-ws74 krb5kdc[10965](info): commencing operation
krb5kdc: starting...
W20260926 06:53:37.546142 767 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:53:37 dist-test-slave-ws74 krb5kdc[10965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:53:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:53:37.659981 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:38499
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:44823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:38499
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:37.737943 10978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:37.738189 10978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:37.738227 10978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:37.739367 10978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:53:37.739430 10978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:37.739455 10978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:37.739477 10978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:53:37.739506 10978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:53:37.740924 10978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:44823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:38499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:38499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:37.741199 10978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:37.741431 10978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:37.746088 10986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:37.746068 10984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:37.746064 10983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:37.746119 10978 server_base.cc:1034] running on GCE node
I20260926 06:53:37.746591 10978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:37.746907 10978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:37.748075 10978 hybrid_clock.cc:648] HybridClock initialized: now 1790405617748062 us; error 29 us; skew 500 ppm
Sep 26 06:53:37 dist-test-slave-ws74 krb5kdc[10965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405617, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:37.750738 10978 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:53:37.751106 10978 webserver.cc:460] Webserver started at http://127.0.191.254:43343/ using document root <none> and password file <none>
I20260926 06:53:37.751336 10978 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:37.751403 10978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:37.751547 10978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:37.752635 10978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "f37c7bdf3e0a4d3ea55a51c76aa04861"
format_stamp: "Formatted at 2026-09-26 06:53:37 on dist-test-slave-ws74"
server_key: "bdc19a9046e58ab2fa81c96a6302eef9"
server_key_iv: "b5eea3411a16582fd94fa162505d09f9"
server_key_version: "encryptionkey@0"
I20260926 06:53:37.752990 10978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "f37c7bdf3e0a4d3ea55a51c76aa04861"
format_stamp: "Formatted at 2026-09-26 06:53:37 on dist-test-slave-ws74"
server_key: "bdc19a9046e58ab2fa81c96a6302eef9"
server_key_iv: "b5eea3411a16582fd94fa162505d09f9"
server_key_version: "encryptionkey@0"
I20260926 06:53:37.754374 10978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:53:37.755141 10993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:37.755303 10978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:53:37.755405 10978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "f37c7bdf3e0a4d3ea55a51c76aa04861"
format_stamp: "Formatted at 2026-09-26 06:53:37 on dist-test-slave-ws74"
server_key: "bdc19a9046e58ab2fa81c96a6302eef9"
server_key_iv: "b5eea3411a16582fd94fa162505d09f9"
server_key_version: "encryptionkey@0"
I20260926 06:53:37.755477 10978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:37.797792 10978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:37.800201 10978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:37.800369 10978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:37.804140 10978 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:38499
I20260926 06:53:37.804225 11045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:38499 every 8 connection(s)
I20260926 06:53:37.804549 10978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:53:37.805559 11046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:37.807750 11046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861: Bootstrap starting.
I20260926 06:53:37.808322 11046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:37.808610 11046 log.cc:824] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:37.809335 11046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861: No bootstrap required, opened a new log
I20260926 06:53:37.810900 11046 raft_consensus.cc:348] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37c7bdf3e0a4d3ea55a51c76aa04861" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38499 } }
I20260926 06:53:37.811033 11046 raft_consensus.cc:374] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:37.811054 11046 raft_consensus.cc:729] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f37c7bdf3e0a4d3ea55a51c76aa04861, State: Initialized, Role: FOLLOWER
I20260926 06:53:37.811235 11046 consensus_queue.cc:260] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37c7bdf3e0a4d3ea55a51c76aa04861" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38499 } }
I20260926 06:53:37.811298 11046 raft_consensus.cc:388] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:53:37.811318 11046 raft_consensus.cc:482] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:53:37.811390 11046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:37.812156 11046 raft_consensus.cc:504] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37c7bdf3e0a4d3ea55a51c76aa04861" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38499 } }
I20260926 06:53:37.812263 11046 leader_election.cc:302] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f37c7bdf3e0a4d3ea55a51c76aa04861; no voters:
I20260926 06:53:37.812443 11046 leader_election.cc:288] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:53:37.812497 11049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:37.812678 11049 raft_consensus.cc:686] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [term 1 LEADER]: Becoming Leader. State: Replica: f37c7bdf3e0a4d3ea55a51c76aa04861, State: Running, Role: LEADER
I20260926 06:53:37.812698 11046 sys_catalog.cc:564] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:53:37.812768 11049 consensus_queue.cc:237] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37c7bdf3e0a4d3ea55a51c76aa04861" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38499 } }
I20260926 06:53:37.813159 11051 sys_catalog.cc:455] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f37c7bdf3e0a4d3ea55a51c76aa04861. Latest consensus state: current_term: 1 leader_uuid: "f37c7bdf3e0a4d3ea55a51c76aa04861" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37c7bdf3e0a4d3ea55a51c76aa04861" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38499 } } }
I20260926 06:53:37.813251 11051 sys_catalog.cc:458] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:37.813155 11050 sys_catalog.cc:455] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f37c7bdf3e0a4d3ea55a51c76aa04861" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37c7bdf3e0a4d3ea55a51c76aa04861" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38499 } } }
I20260926 06:53:37.813498 11050 sys_catalog.cc:458] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861 [sys.catalog]: This master's current role is: LEADER
I20260926 06:53:37.814029 11058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:53:37.814446 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 10978
I20260926 06:53:37.814553 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:53:37.814574 11058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:53:37.814764 767 external_mini_cluster.cc:1428] Setting key 97ebb0ba6ccfa098d0abe3404928c4d3
I20260926 06:53:37.816105 11058 catalog_manager.cc:1347] Generated new cluster ID: c42953ed463346d98f87e98e89d477a4
I20260926 06:53:37.816160 11058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:53:37 dist-test-slave-ws74 krb5kdc[10965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:37.826180 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:37.815321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40024 (local address 127.0.191.254:38499)
0926 06:53:37.815469 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:37.815477 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:37.815506 (+ 29us) server_negotiation.cc:407] Connection header received
0926 06:53:37.815612 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:37.815619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:37.815683 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:37.815812 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:37.816669 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:37.818022 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:37.819101 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:37.819689 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:37.819799 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:37.822555 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:37.822559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:37.822559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:37.822570 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:37.824554 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:37.824927 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:37.824930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:37.824930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:37.825043 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:37.825240 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:37.825241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:37.825241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:37.825579 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:37.825727 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:37.825796 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 06:53:37.825946 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260926 06:53:37.837208 11058 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:53:37.837905 11058 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:53:37.845300 11058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861: Generated new TSK 0
I20260926 06:53:37.845482 11058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:53:37.876389 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38499
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:44823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:37.955749 11072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:37.955924 11072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:37.955946 11072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:37.956947 11072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:37.956979 11072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:37.957003 11072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:53:37.958482 11072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:44823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:37.958655 11072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:37.958817 11072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:37.963861 11077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:37.963896 11078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:37.963878 11072 server_base.cc:1034] running on GCE node
W20260926 06:53:37.963857 11080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:37.964237 11072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:37.964478 11072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:37.965632 11072 hybrid_clock.cc:648] HybridClock initialized: now 1790405617965623 us; error 48 us; skew 500 ppm
Sep 26 06:53:37 dist-test-slave-ws74 krb5kdc[10965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405617, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:37.967954 11072 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:53:37.968415 11072 webserver.cc:460] Webserver started at http://127.0.191.193:35499/ using document root <none> and password file <none>
I20260926 06:53:37.968647 11072 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:37.968711 11072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:37.968854 11072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:37.970139 11072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "046d0440bcac47b9b364c1e7514bf8fa"
format_stamp: "Formatted at 2026-09-26 06:53:37 on dist-test-slave-ws74"
server_key: "92f9d413e9ddf65c611f4ec082aa5d5c"
server_key_iv: "c197d8bb1802532fd77362c99e8fda5c"
server_key_version: "encryptionkey@0"
I20260926 06:53:37.970543 11072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "046d0440bcac47b9b364c1e7514bf8fa"
format_stamp: "Formatted at 2026-09-26 06:53:37 on dist-test-slave-ws74"
server_key: "92f9d413e9ddf65c611f4ec082aa5d5c"
server_key_iv: "c197d8bb1802532fd77362c99e8fda5c"
server_key_version: "encryptionkey@0"
I20260926 06:53:37.971872 11072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:53:37.972497 11087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:37.972657 11072 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:37.972744 11072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "046d0440bcac47b9b364c1e7514bf8fa"
format_stamp: "Formatted at 2026-09-26 06:53:37 on dist-test-slave-ws74"
server_key: "92f9d413e9ddf65c611f4ec082aa5d5c"
server_key_iv: "c197d8bb1802532fd77362c99e8fda5c"
server_key_version: "encryptionkey@0"
I20260926 06:53:37.972796 11072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:37.987174 11072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:37.988919 11072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:37.989025 11072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:37.989193 11072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:37.989485 11072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:37.989521 11072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:37.989542 11072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:37.989553 11072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:37.994999 11072 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:42431
I20260926 06:53:37.995079 11200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:42431 every 8 connection(s)
I20260926 06:53:37.995368 11072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:53:38.000795 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 11072
I20260926 06:53:38.000877 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:53:38.001080 767 external_mini_cluster.cc:1428] Setting key b8d3fe39c3f7dc764b3564eaa8807776
Sep 26 06:53:38 dist-test-slave-ws74 krb5kdc[10965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405617, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:38.005255 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:37.996063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:51963 (local address 127.0.191.254:38499)
0926 06:53:37.996173 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:37.996175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:37.996357 (+ 182us) server_negotiation.cc:407] Connection header received
0926 06:53:37.996902 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:37.996903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:37.996915 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:37.996974 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:37.997932 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:37.998443 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:37.999457 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:37.999675 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:37.999783 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.002130 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:38.002132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:38.002132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:38.002140 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:38.003814 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:38.004211 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:38.004212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:38.004212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:38.004263 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:38.004547 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:38.004548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:38.004549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:38.004669 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:38.004748 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:38.005010 (+ 262us) server_negotiation.cc:299] Negotiation successful
0926 06:53:38.005106 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 06:53:38.005376 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:37.996023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:38499 (local address 127.0.191.193:51963)
0926 06:53:37.996226 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:37.996269 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:37.996788 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:37.997044 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:37.997049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:37.997627 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:37.997810 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:37.997814 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:37.998538 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:37.998539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:37.999356 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:37.999358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:37.999791 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:37.999792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:37.999918 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.000429 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:38.000437 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:38.002015 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:38.003904 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:38.003910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:38.003912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:38.004139 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:38.004362 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:38.004363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:38.004363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:38.004476 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:38.004740 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:38.004746 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:38.004882 (+ 136us) client_negotiation.cc:769] Sending connection context
0926 06:53:38.004989 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 06:53:38.005150 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":118,"threads_started":1}
I20260926 06:53:38.005936 11201 heartbeater.cc:347] Connected to a master server at 127.0.191.254:38499
I20260926 06:53:38.006037 11201 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:38.006250 11201 heartbeater.cc:511] Master 127.0.191.254:38499 requested a full tablet report, sending...
I20260926 06:53:38.006774 11010 ts_manager.cc:194] Registered new tserver with Master: 046d0440bcac47b9b364c1e7514bf8fa (127.0.191.193:42431)
I20260926 06:53:38.007376 11010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:51963
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:53:38.028600 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38499
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:44823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:38.108062 11206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:38.108259 11206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:38.108281 11206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:38.109323 11206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:38.109359 11206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:38.109447 11206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:53:38.110793 11206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:44823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:38.110960 11206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:38.111119 11206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:38.116096 11211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:38.116235 11214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:38.116407 11212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:38.116421 11206 server_base.cc:1034] running on GCE node
I20260926 06:53:38.116663 11206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:38.116917 11206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:38.118075 11206 hybrid_clock.cc:648] HybridClock initialized: now 1790405618118065 us; error 39 us; skew 500 ppm
Sep 26 06:53:38 dist-test-slave-ws74 krb5kdc[10965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405618, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:38.120414 11206 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:53:38.120846 11206 webserver.cc:460] Webserver started at http://127.0.191.194:41431/ using document root <none> and password file <none>
I20260926 06:53:38.121034 11206 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:38.121073 11206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:38.121165 11206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:38.122169 11206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "748aaa57c6ff4986855f8281e993ca7e"
format_stamp: "Formatted at 2026-09-26 06:53:38 on dist-test-slave-ws74"
server_key: "a51cff68d80cf7a2415a0d7defaf6344"
server_key_iv: "700dafca4d5fa9efb7d6017ae4b6744b"
server_key_version: "encryptionkey@0"
I20260926 06:53:38.122447 11206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "748aaa57c6ff4986855f8281e993ca7e"
format_stamp: "Formatted at 2026-09-26 06:53:38 on dist-test-slave-ws74"
server_key: "a51cff68d80cf7a2415a0d7defaf6344"
server_key_iv: "700dafca4d5fa9efb7d6017ae4b6744b"
server_key_version: "encryptionkey@0"
I20260926 06:53:38.123624 11206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:53:38.124266 11221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:38.124495 11206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:53:38.124601 11206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "748aaa57c6ff4986855f8281e993ca7e"
format_stamp: "Formatted at 2026-09-26 06:53:38 on dist-test-slave-ws74"
server_key: "a51cff68d80cf7a2415a0d7defaf6344"
server_key_iv: "700dafca4d5fa9efb7d6017ae4b6744b"
server_key_version: "encryptionkey@0"
I20260926 06:53:38.124672 11206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:38.136677 11206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:38.138573 11206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:38.138733 11206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:38.138960 11206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:38.139259 11206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:38.139292 11206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:38.139314 11206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:38.139361 11206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:38.145308 11206 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:39931
I20260926 06:53:38.145349 11334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:39931 every 8 connection(s)
I20260926 06:53:38.145680 11206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:53:38 dist-test-slave-ws74 krb5kdc[10965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405618, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:38.153167 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 11206
I20260926 06:53:38.153277 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:53:38.153523 767 external_mini_cluster.cc:1428] Setting key 8f36d542f226dd886b702757c585496e
I20260926 06:53:38.155216 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.146383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:56469 (local address 127.0.191.254:38499)
0926 06:53:38.146433 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:38.146436 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:38.146651 (+ 215us) server_negotiation.cc:407] Connection header received
0926 06:53:38.147128 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:38.147129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:38.147140 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:38.147231 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:38.148112 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.148622 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.149659 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.149848 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.149911 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.152042 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:38.152044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:38.152044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:38.152050 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:38.153665 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:38.154176 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:38.154177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:38.154178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:38.154243 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:38.154537 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:38.154538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:38.154538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:38.154695 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:38.154766 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:38.154995 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 06:53:38.155086 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:53:38.155246 11337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.146373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:38499 (local address 127.0.191.194:56469)
0926 06:53:38.146532 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:38.146567 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:38.147019 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:38.147303 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:38.147306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:38.147844 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:38.148022 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.148026 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.148704 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.148705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.149570 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.149573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.149937 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.149938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.150038 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.150655 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:38.150662 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:38.151961 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:38.153767 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:38.153775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:38.153776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:38.154063 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:38.154316 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:38.154317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:38.154318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:38.154449 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:38.154760 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:38.154764 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:38.154872 (+ 108us) client_negotiation.cc:769] Sending connection context
0926 06:53:38.154953 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 06:53:38.155067 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260926 06:53:38.155701 11335 heartbeater.cc:347] Connected to a master server at 127.0.191.254:38499
I20260926 06:53:38.155812 11335 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:38.155992 11335 heartbeater.cc:511] Master 127.0.191.254:38499 requested a full tablet report, sending...
I20260926 06:53:38.156379 11010 ts_manager.cc:194] Registered new tserver with Master: 748aaa57c6ff4986855f8281e993ca7e (127.0.191.194:39931)
I20260926 06:53:38.156749 11010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:56469
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:53:38.179992 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38499
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:44823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:53:38.259984 11340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:53:38.260176 11340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:53:38.260198 11340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:53:38.261298 11340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:53:38.261338 11340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:53:38.261364 11340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:53:38.262800 11340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:44823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:53:38.263029 11340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:53:38.263278 11340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:53:38.268107 11348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:38.268124 11345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:53:38.268263 11346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:53:38.268507 11340 server_base.cc:1034] running on GCE node
I20260926 06:53:38.268693 11340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:53:38.268872 11340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:53:38.270004 11340 hybrid_clock.cc:648] HybridClock initialized: now 1790405618269975 us; error 44 us; skew 500 ppm
Sep 26 06:53:38 dist-test-slave-ws74 krb5kdc[10965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405618, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:53:38.272425 11340 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:53:38.272866 11340 webserver.cc:460] Webserver started at http://127.0.191.195:38023/ using document root <none> and password file <none>
I20260926 06:53:38.273077 11340 fs_manager.cc:359] Metadata directory not provided
I20260926 06:53:38.273140 11340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:53:38.273312 11340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:53:38.274333 11340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "2757a14fb98442e38b974e18a8b4872e"
format_stamp: "Formatted at 2026-09-26 06:53:38 on dist-test-slave-ws74"
server_key: "4ebf6e4454a8a1ef72c5b14784a65de3"
server_key_iv: "ba0cf3d4c0fa459c3bcb27fa45e2993b"
server_key_version: "encryptionkey@0"
I20260926 06:53:38.274673 11340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "2757a14fb98442e38b974e18a8b4872e"
format_stamp: "Formatted at 2026-09-26 06:53:38 on dist-test-slave-ws74"
server_key: "4ebf6e4454a8a1ef72c5b14784a65de3"
server_key_iv: "ba0cf3d4c0fa459c3bcb27fa45e2993b"
server_key_version: "encryptionkey@0"
I20260926 06:53:38.276033 11340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:53:38.276746 11355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:38.276906 11340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:53:38.277002 11340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "2757a14fb98442e38b974e18a8b4872e"
format_stamp: "Formatted at 2026-09-26 06:53:38 on dist-test-slave-ws74"
server_key: "4ebf6e4454a8a1ef72c5b14784a65de3"
server_key_iv: "ba0cf3d4c0fa459c3bcb27fa45e2993b"
server_key_version: "encryptionkey@0"
I20260926 06:53:38.277083 11340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:53:38.291532 11340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:53:38.293355 11340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:53:38.293560 11340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:53:38.293777 11340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:53:38.294070 11340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:53:38.294130 11340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:38.294171 11340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:53:38.294201 11340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:53:38.301615 11340 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:42803
I20260926 06:53:38.301690 11468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:42803 every 8 connection(s)
I20260926 06:53:38.302126 11340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:53:38.304591 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 11340
I20260926 06:53:38.304689 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:53:38.304900 767 external_mini_cluster.cc:1428] Setting key 6495446e7e828bc558ef9b6dae8c77c9
Sep 26 06:53:38 dist-test-slave-ws74 krb5kdc[10965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405618, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:53:38.312768 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.302934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:34195 (local address 127.0.191.254:38499)
0926 06:53:38.303003 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:38.303006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:38.303295 (+ 289us) server_negotiation.cc:407] Connection header received
0926 06:53:38.304537 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:38.304539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:38.304553 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:38.304621 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:38.305452 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.306255 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.307711 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.307928 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.308015 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.310223 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:38.310225 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:38.310225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:38.310231 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:38.311440 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:38.311799 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:38.311800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:38.311801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:38.311854 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:38.312129 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:38.312130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:38.312130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:38.312233 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:38.312372 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:38.312644 (+ 272us) server_negotiation.cc:299] Negotiation successful
0926 06:53:38.312705 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 06:53:38.312923 11471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.302897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:38499 (local address 127.0.191.195:34195)
0926 06:53:38.303096 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:38.303190 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:38.304402 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:38.304719 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:38.304723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:38.305035 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:53:38.305355 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.305362 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.306368 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.306369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.307613 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.307616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.308025 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.308026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.308164 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.308615 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:53:38.308625 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:53:38.310106 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:53:38.311511 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:38.311516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:38.311518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:38.311729 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:38.311915 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:53:38.311916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:53:38.311916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:53:38.312068 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:53:38.312366 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:53:38.312370 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:53:38.312526 (+ 156us) client_negotiation.cc:769] Sending connection context
0926 06:53:38.312612 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 06:53:38.312729 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260926 06:53:38.313519 11469 heartbeater.cc:347] Connected to a master server at 127.0.191.254:38499
I20260926 06:53:38.313633 11469 heartbeater.cc:464] Registering TS with master...
I20260926 06:53:38.313812 11469 heartbeater.cc:511] Master 127.0.191.254:38499 requested a full tablet report, sending...
I20260926 06:53:38.314261 11010 ts_manager.cc:194] Registered new tserver with Master: 2757a14fb98442e38b974e18a8b4872e (127.0.191.195:42803)
I20260926 06:53:38.314653 11010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:34195
I20260926 06:53:38.316686 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:53:38.323516 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.317989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40028 (local address 127.0.191.254:38499)
0926 06:53:38.318044 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:38.318046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:38.318175 (+ 129us) server_negotiation.cc:407] Connection header received
0926 06:53:38.318197 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:38.318198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:38.318206 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:38.318256 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:53:38.318751 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.319193 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.319902 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.320069 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.320116 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.321342 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:53:38.321343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:53:38.321343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:53:38.321348 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:53:38.322537 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:38.322858 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:38.322859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:38.322859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:38.322912 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:53:38.323093 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:53:38.323094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:53:38.323094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:53:38.323217 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:53:38.323292 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:38.323375 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 06:53:38.323430 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:53:38.325093 11010 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:53:38.326699 11010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:53:38.333591 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.328923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:39931 (local address 127.0.0.1:51694)
0926 06:53:38.329231 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:38.329271 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:38.329337 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:38.329924 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:38.329924 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:38.329926 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:38.329953 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.329954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.331150 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.331151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.332409 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.332411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.333267 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.333268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.333331 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.333341 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 06:53:38.333469 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 06:53:38.333521 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":71,"thread_start_us":55,"threads_started":1}
I20260926 06:53:38.333602 11481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.329084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51694 (local address 127.0.191.194:39931)
0926 06:53:38.329780 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:38.329784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:38.329793 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:53:38.329829 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:38.329831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:38.329846 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:38.329923 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:38.329996 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.331020 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.332497 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.333190 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.333261 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.333312 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:38.333488 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 06:53:38.333528 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":116,"threads_started":1}
I20260926 06:53:38.334388 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.328922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:42431 (local address 127.0.0.1:41492)
0926 06:53:38.329229 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:38.329269 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:38.329335 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:38.329644 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:38.329647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:38.329649 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:38.329732 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.329734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.331068 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.331068 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.332409 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.332411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.334177 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.334178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.334238 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.334242 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:53:38.334302 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 06:53:38.334346 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":53,"threads_started":1}
I20260926 06:53:38.334645 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.329023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41492 (local address 127.0.191.193:42431)
0926 06:53:38.329499 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:38.329504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:38.329514 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:53:38.329553 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:38.329554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:38.329567 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:38.329658 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:38.329843 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.330967 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.333346 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.334096 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.334163 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.334213 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:38.334508 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 06:53:38.334549 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":115,"threads_started":1}
I20260926 06:53:38.334705 11269 tablet_service.cc:1467] Processing CreateTablet for tablet 27fbd551e07c41698c84db480c29b5f0 (DEFAULT_TABLE table=test-table [id=00c436af2adb418b843032088b95d253]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:38.335076 11269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27fbd551e07c41698c84db480c29b5f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:38.335230 11135 tablet_service.cc:1467] Processing CreateTablet for tablet 27fbd551e07c41698c84db480c29b5f0 (DEFAULT_TABLE table=test-table [id=00c436af2adb418b843032088b95d253]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:38.335474 11135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27fbd551e07c41698c84db480c29b5f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:38.337719 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.329111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:42803 (local address 127.0.0.1:50216)
0926 06:53:38.329274 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:38.329284 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:38.329335 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:38.331314 (+ 1979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:38.331315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:38.331317 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:38.331385 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.331387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.332380 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.332381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.333450 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.333453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.337454 (+ 4001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.337456 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.337529 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.337534 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:53:38.337599 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 06:53:38.337644 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":22,"thread_start_us":57,"threads_started":1}
I20260926 06:53:38.337770 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.329202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50216 (local address 127.0.191.195:42803)
0926 06:53:38.331175 (+ 1973us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:38.331178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:38.331186 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:53:38.331212 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:38.331213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:38.331225 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:38.331315 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:38.331447 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.332283 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.333534 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.334228 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.334303 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.334348 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:38.337633 (+ 3285us) server_negotiation.cc:299] Negotiation successful
0926 06:53:38.337679 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1945,"thread_start_us":99,"threads_started":1}
I20260926 06:53:38.337945 11483 tablet_bootstrap.cc:492] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e: Bootstrap starting.
I20260926 06:53:38.338155 11484 tablet_bootstrap.cc:492] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa: Bootstrap starting.
I20260926 06:53:38.338374 11394 tablet_service.cc:1467] Processing CreateTablet for tablet 27fbd551e07c41698c84db480c29b5f0 (DEFAULT_TABLE table=test-table [id=00c436af2adb418b843032088b95d253]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:53:38.338608 11483 tablet_bootstrap.cc:654] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:38.338685 11394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27fbd551e07c41698c84db480c29b5f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:53:38.338763 11484 tablet_bootstrap.cc:654] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:38.339157 11483 log.cc:824] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:38.339306 11484 log.cc:824] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:38.339979 11483 tablet_bootstrap.cc:492] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e: No bootstrap required, opened a new log
I20260926 06:53:38.340018 11484 tablet_bootstrap.cc:492] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa: No bootstrap required, opened a new log
I20260926 06:53:38.340063 11483 ts_tablet_manager.cc:1397] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:53:38.340095 11484 ts_tablet_manager.cc:1397] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:53:38.341491 11484 raft_consensus.cc:348] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "748aaa57c6ff4986855f8281e993ca7e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 39931 } } peers { permanent_uuid: "046d0440bcac47b9b364c1e7514bf8fa" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42431 } } peers { permanent_uuid: "2757a14fb98442e38b974e18a8b4872e" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42803 } }
I20260926 06:53:38.341678 11484 raft_consensus.cc:374] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:38.341722 11484 raft_consensus.cc:729] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 046d0440bcac47b9b364c1e7514bf8fa, State: Initialized, Role: FOLLOWER
I20260926 06:53:38.341845 11484 consensus_queue.cc:260] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "748aaa57c6ff4986855f8281e993ca7e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 39931 } } peers { permanent_uuid: "046d0440bcac47b9b364c1e7514bf8fa" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42431 } } peers { permanent_uuid: "2757a14fb98442e38b974e18a8b4872e" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42803 } }
I20260926 06:53:38.342135 11484 ts_tablet_manager.cc:1428] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:53:38.342136 11487 tablet_bootstrap.cc:492] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e: Bootstrap starting.
I20260926 06:53:38.342263 11201 heartbeater.cc:503] Master 127.0.191.254:38499 was elected leader, sending a full tablet report...
I20260926 06:53:38.342748 11487 tablet_bootstrap.cc:654] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e: Neither blocks nor log segments found. Creating new log.
I20260926 06:53:38.343034 11483 raft_consensus.cc:348] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "748aaa57c6ff4986855f8281e993ca7e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 39931 } } peers { permanent_uuid: "046d0440bcac47b9b364c1e7514bf8fa" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42431 } } peers { permanent_uuid: "2757a14fb98442e38b974e18a8b4872e" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42803 } }
I20260926 06:53:38.343200 11483 raft_consensus.cc:374] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:38.343243 11483 raft_consensus.cc:729] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 748aaa57c6ff4986855f8281e993ca7e, State: Initialized, Role: FOLLOWER
I20260926 06:53:38.343348 11487 log.cc:824] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:53:38.343355 11483 consensus_queue.cc:260] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "748aaa57c6ff4986855f8281e993ca7e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 39931 } } peers { permanent_uuid: "046d0440bcac47b9b364c1e7514bf8fa" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42431 } } peers { permanent_uuid: "2757a14fb98442e38b974e18a8b4872e" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42803 } }
I20260926 06:53:38.343629 11483 ts_tablet_manager.cc:1428] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260926 06:53:38.343647 11335 heartbeater.cc:503] Master 127.0.191.254:38499 was elected leader, sending a full tablet report...
I20260926 06:53:38.344437 11487 tablet_bootstrap.cc:492] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e: No bootstrap required, opened a new log
I20260926 06:53:38.344507 11487 ts_tablet_manager.cc:1397] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:53:38.345913 11487 raft_consensus.cc:348] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "748aaa57c6ff4986855f8281e993ca7e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 39931 } } peers { permanent_uuid: "046d0440bcac47b9b364c1e7514bf8fa" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42431 } } peers { permanent_uuid: "2757a14fb98442e38b974e18a8b4872e" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42803 } }
I20260926 06:53:38.346062 11487 raft_consensus.cc:374] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:53:38.346107 11487 raft_consensus.cc:729] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2757a14fb98442e38b974e18a8b4872e, State: Initialized, Role: FOLLOWER
I20260926 06:53:38.346217 11487 consensus_queue.cc:260] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "748aaa57c6ff4986855f8281e993ca7e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 39931 } } peers { permanent_uuid: "046d0440bcac47b9b364c1e7514bf8fa" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42431 } } peers { permanent_uuid: "2757a14fb98442e38b974e18a8b4872e" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42803 } }
I20260926 06:53:38.346452 11487 ts_tablet_manager.cc:1428] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:53:38.346495 11469 heartbeater.cc:503] Master 127.0.191.254:38499 was elected leader, sending a full tablet report...
W20260926 06:53:38.396286 11336 tablet.cc:2369] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:53:38.456946 11491 raft_consensus.cc:482] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:53:38.457103 11491 raft_consensus.cc:504] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "748aaa57c6ff4986855f8281e993ca7e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 39931 } } peers { permanent_uuid: "046d0440bcac47b9b364c1e7514bf8fa" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42431 } } peers { permanent_uuid: "2757a14fb98442e38b974e18a8b4872e" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42803 } }
I20260926 06:53:38.457504 11491 leader_election.cc:288] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 748aaa57c6ff4986855f8281e993ca7e (127.0.191.194:39931), 046d0440bcac47b9b364c1e7514bf8fa (127.0.191.193:42431)
I20260926 06:53:38.460594 11471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.457668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:39931 (local address 127.0.191.195:45363)
0926 06:53:38.457736 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:38.457748 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:38.457794 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:38.457975 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:38.457976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:38.457978 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:38.458056 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.458064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.458860 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.458861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.459704 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.459706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.460343 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.460344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.460423 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.460428 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:53:38.460480 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 06:53:38.460531 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 06:53:38.460613 11481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.457764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:45363 (local address 127.0.191.194:39931)
0926 06:53:38.457810 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:38.457812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:38.457824 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:53:38.457855 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:38.457856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:38.457865 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:38.457966 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:38.458190 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.458765 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.459792 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.460253 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.460311 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.460318 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:38.460503 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 06:53:38.460539 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 06:53:38.460775 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.457680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:42431 (local address 127.0.191.195:53201)
0926 06:53:38.457859 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:38.457869 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:53:38.457959 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:53:38.458189 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:53:38.458190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:53:38.458191 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:53:38.458265 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.458267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.458945 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.458946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.459892 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:53:38.459894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.460506 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.460507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:53:38.460572 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.460576 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:53:38.460642 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 06:53:38.460684 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":103,"threads_started":1}
I20260926 06:53:38.460789 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.457784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:53201 (local address 127.0.191.193:42431)
0926 06:53:38.457831 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:38.457833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:38.457947 (+ 114us) server_negotiation.cc:407] Connection header received
0926 06:53:38.458077 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:38.458079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:38.458089 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:38.458146 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:53:38.458338 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.458836 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.459982 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.460410 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.460487 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.460494 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:38.460667 (+ 173us) server_negotiation.cc:299] Negotiation successful
0926 06:53:38.460714 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:53:38.461016 11289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27fbd551e07c41698c84db480c29b5f0" candidate_uuid: "2757a14fb98442e38b974e18a8b4872e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "748aaa57c6ff4986855f8281e993ca7e" is_pre_election: true
I20260926 06:53:38.461167 11289 raft_consensus.cc:2454] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2757a14fb98442e38b974e18a8b4872e in term 0.
I20260926 06:53:38.461174 11155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27fbd551e07c41698c84db480c29b5f0" candidate_uuid: "2757a14fb98442e38b974e18a8b4872e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "046d0440bcac47b9b364c1e7514bf8fa" is_pre_election: true
I20260926 06:53:38.461328 11155 raft_consensus.cc:2454] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2757a14fb98442e38b974e18a8b4872e in term 0.
I20260926 06:53:38.461428 11357 leader_election.cc:302] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2757a14fb98442e38b974e18a8b4872e, 748aaa57c6ff4986855f8281e993ca7e; no voters:
I20260926 06:53:38.461552 11491 raft_consensus.cc:2781] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:53:38.461606 11491 raft_consensus.cc:482] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:53:38.461640 11491 raft_consensus.cc:3037] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:38.462332 11491 raft_consensus.cc:504] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "748aaa57c6ff4986855f8281e993ca7e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 39931 } } peers { permanent_uuid: "046d0440bcac47b9b364c1e7514bf8fa" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42431 } } peers { permanent_uuid: "2757a14fb98442e38b974e18a8b4872e" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42803 } }
I20260926 06:53:38.462487 11491 leader_election.cc:288] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [CANDIDATE]: Term 1 election: Requested vote from peers 748aaa57c6ff4986855f8281e993ca7e (127.0.191.194:39931), 046d0440bcac47b9b364c1e7514bf8fa (127.0.191.193:42431)
I20260926 06:53:38.462649 11289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27fbd551e07c41698c84db480c29b5f0" candidate_uuid: "2757a14fb98442e38b974e18a8b4872e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "748aaa57c6ff4986855f8281e993ca7e"
I20260926 06:53:38.462704 11155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27fbd551e07c41698c84db480c29b5f0" candidate_uuid: "2757a14fb98442e38b974e18a8b4872e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "046d0440bcac47b9b364c1e7514bf8fa"
I20260926 06:53:38.462749 11289 raft_consensus.cc:3037] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:38.462783 11155 raft_consensus.cc:3037] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:53:38.463518 11155 raft_consensus.cc:2454] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2757a14fb98442e38b974e18a8b4872e in term 1.
I20260926 06:53:38.463625 11289 raft_consensus.cc:2454] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2757a14fb98442e38b974e18a8b4872e in term 1.
I20260926 06:53:38.463694 11356 leader_election.cc:302] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 046d0440bcac47b9b364c1e7514bf8fa, 2757a14fb98442e38b974e18a8b4872e; no voters:
I20260926 06:53:38.463791 11491 raft_consensus.cc:2781] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:53:38.463944 11491 raft_consensus.cc:686] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [term 1 LEADER]: Becoming Leader. State: Replica: 2757a14fb98442e38b974e18a8b4872e, State: Running, Role: LEADER
I20260926 06:53:38.464042 11491 consensus_queue.cc:237] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "748aaa57c6ff4986855f8281e993ca7e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 39931 } } peers { permanent_uuid: "046d0440bcac47b9b364c1e7514bf8fa" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42431 } } peers { permanent_uuid: "2757a14fb98442e38b974e18a8b4872e" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42803 } }
I20260926 06:53:38.464911 11010 catalog_manager.cc:5495] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2757a14fb98442e38b974e18a8b4872e (127.0.191.195). New cstate: current_term: 1 leader_uuid: "2757a14fb98442e38b974e18a8b4872e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "748aaa57c6ff4986855f8281e993ca7e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 39931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "046d0440bcac47b9b364c1e7514bf8fa" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2757a14fb98442e38b974e18a8b4872e" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 42803 } health_report { overall_health: HEALTHY } } }
I20260926 06:53:38.478567 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.476499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.0.191.195:42803)
0926 06:53:38.476536 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:53:38.476538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:53:38.476546 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:53:38.476583 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:53:38.476587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:53:38.476603 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:53:38.476667 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:53:38.476795 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.477235 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.477967 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:53:38.478142 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:53:38.478192 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:53:38.478321 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:53:38.478405 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:53:38.478463 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 06:53:38.478492 (+ 29us) server_negotiation.cc:299] Negotiation successful
0926 06:53:38.478506 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:53:38.480252 11289 raft_consensus.cc:1261] T 27fbd551e07c41698c84db480c29b5f0 P 748aaa57c6ff4986855f8281e993ca7e [term 1 FOLLOWER]: Refusing update from remote peer 2757a14fb98442e38b974e18a8b4872e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:38.480252 11155 raft_consensus.cc:1261] T 27fbd551e07c41698c84db480c29b5f0 P 046d0440bcac47b9b364c1e7514bf8fa [term 1 FOLLOWER]: Refusing update from remote peer 2757a14fb98442e38b974e18a8b4872e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:53:38.480574 11491 consensus_queue.cc:1035] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [LEADER]: Connected to new peer: Peer: permanent_uuid: "748aaa57c6ff4986855f8281e993ca7e" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 39931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:38.480705 11493 consensus_queue.cc:1035] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [LEADER]: Connected to new peer: Peer: permanent_uuid: "046d0440bcac47b9b364c1e7514bf8fa" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:53:38.482954 11496 mvcc.cc:205] Tried to move back new op lower bound from 7333501413292642304 to 7333501413229359104. Current Snapshot: MvccSnapshot[applied={T|T < 7333501413292642304}]
I20260926 06:53:38.483927 11497 mvcc.cc:205] Tried to move back new op lower bound from 7333501413292642304 to 7333501413229359104. Current Snapshot: MvccSnapshot[applied={T|T < 7333501413292642304 or (T in {7333501413292642304})}]
I20260926 06:53:38.485659 11009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:38.485754 11009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:53:38.486469 11009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f37c7bdf3e0a4d3ea55a51c76aa04861: Sending DeleteTablet for 3 replicas of tablet 27fbd551e07c41698c84db480c29b5f0
I20260926 06:53:38.486778 11269 tablet_service.cc:1514] Processing DeleteTablet for tablet 27fbd551e07c41698c84db480c29b5f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:38 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:51694
I20260926 06:53:38.486791 11394 tablet_service.cc:1514] Processing DeleteTablet for tablet 27fbd551e07c41698c84db480c29b5f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:38 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:50216
I20260926 06:53:38.486796 11135 tablet_service.cc:1514] Processing DeleteTablet for tablet 27fbd551e07c41698c84db480c29b5f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:53:38 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:41492
I20260926 06:53:38.487036 11507 tablet_replica.cc:330] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e: stopping tablet replica
I20260926 06:53:38.487140 11507 raft_consensus.cc:2229] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:53:38.487250 11507 raft_consensus.cc:2258] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:53:38.487275 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 11072
I20260926 06:53:38.488358 11507 ts_tablet_manager.cc:1905] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:53:38.489936 11507 ts_tablet_manager.cc:1918] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:53:38.490007 11507 log.cc:1196] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/27fbd551e07c41698c84db480c29b5f0
I20260926 06:53:38.490288 11507 ts_tablet_manager.cc:1939] T 27fbd551e07c41698c84db480c29b5f0 P 2757a14fb98442e38b974e18a8b4872e: Deleting consensus metadata
I20260926 06:53:38.490792 10995 catalog_manager.cc:4841] TS 2757a14fb98442e38b974e18a8b4872e (127.0.191.195:42803): tablet 27fbd551e07c41698c84db480c29b5f0 (table test-table [id=00c436af2adb418b843032088b95d253]) successfully deleted
W20260926 06:53:38.492185 10994 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:42431 (error 108)
I20260926 06:53:38.492683 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.492553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:42431 (local address 127.0.0.1:41500)
0926 06:53:38.492612 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:38.492651 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:42431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 06:53:38.492794 10994 catalog_manager.cc:4584] TS 046d0440bcac47b9b364c1e7514bf8fa (127.0.191.193:42431): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 27fbd551e07c41698c84db480c29b5f0: Network error: Client connection negotiation failed: client connection to 127.0.191.193:42431: connect: Connection refused (error 111)
I20260926 06:53:38.493593 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 11206
I20260926 06:53:38.499958 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 11340
I20260926 06:53:38.499991 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:53:38.499871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:39931 (local address 127.0.0.1:51696)
0926 06:53:38.499927 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 06:53:38.499960 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.194:39931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 06:53:38.503778 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 10978
2026-09-26T06:53:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1992 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:53:38 dist-test-slave-ws74 krb5kdc[11512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:53:38 dist-test-slave-ws74 krb5kdc[11512](info): set up 2 sockets
Sep 26 06:53:38 dist-test-slave-ws74 krb5kdc[11512](info): commencing operation
krb5kdc: starting...
W20260926 06:53:39.538319 767 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:53:39 dist-test-slave-ws74 krb5kdc[11512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:53:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:53:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.191.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.191.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/rangeradmin_127.0.191.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/rangeradmin_127.0.191.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.191.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.191.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/rangerlookup_127.0.191.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/rangerlookup_127.0.191.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.191.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.191.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/HTTP_127.0.191.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/HTTP_127.0.191.212@KRBTEST.COM.keytab.
I20260926 06:53:39.695462 767 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskyAhorJ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/postgres -l logfile start
2026-09-26 06:53:42.130 UTC [11542] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 06:53:42.130 UTC [11542] LOG: listening on IPv4 address "127.0.191.212", port 51059
2026-09-26 06:53:42.134 UTC [11542] LOG: listening on Unix socket "/tmp/.s.PGSQL.51059"
2026-09-26 06:53:42.154 UTC [11545] LOG: database system was shut down at 2026-09-26 06:53:40 UTC
2026-09-26 06:53:42.158 UTC [11542] LOG: database system is ready to accept connections
I20260926 06:53:43.126430 767 mini_postgres.cc:96] Postgres bound to 51059
2026-09-26 06:53:43.132 UTC [11553] FATAL: database "slave" does not exist
127.0.191.212:51059 - accepting connections
I20260926 06:53:43.132850 767 mini_ranger.cc:161] Starting Ranger...
I20260926 06:53:43.147013 767 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 06:53:43.622613 767 mini_ranger.cc:87] Created ranger Postgres database
I20260926 06:53:43.622699 767 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-admin
2026-09-26 06:53:43,925 [I] DB FLAVOR :POSTGRES
2026-09-26 06:53:43,926 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 06:53:43,926 [I] Checking connection..
2026-09-26 06:53:43,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 06:53:44,164 [I] Checking connection passed.
2026-09-26 06:53:44,165 [I] --------- Verifying version history table ---------
2026-09-26 06:53:44,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 06:53:44,419 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 06:53:44,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 06:53:44,673 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 06:53:44,673 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 06:53:44,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 06:53:44.902 UTC [11617] WARNING: there is no transaction in progress
2026-09-26 06:53:44,908 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 06:53:44,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 06:53:45,162 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 06:53:45,162 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 06:53:45,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 06:53:45,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 06:53:45,666 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ws74.c.gcp-upstream.internal','N') ;"
2026-09-26 06:53:45,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 06:53:46,131 [I] Table x_portal_user does not exist in database ranger
2026-09-26 06:53:46,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 06:53:46,356 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 06:53:46,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 06:53:46,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 06:53:46,858 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 06:53:46,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 06:53:47.797 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:47.812 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:47.829 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:47.844 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:47.859 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:47.876 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:47.930 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:47.936 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:47.949 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:47.959 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:47.969 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:47.983 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:47.993 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:47.999 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:48.006 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:48.013 UTC [11725] WARNING: there is no transaction in progress
2026-09-26 06:53:48,639 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 06:53:48,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 06:53:48,869 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 06:53:48,870 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 06:53:49,122 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 06:53:49,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 06:53:49,346 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 06:53:49,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 06:53:49,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 06:53:49,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ws74.c.gcp-upstream.internal';"
2026-09-26 06:53:50,035 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 06:53:50,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 06:53:50,287 [I] DB_PATCHES have already been applied
I20260926 06:53:50.292946 767 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-admin:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskyAhorJ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 06:53:50.293036 767 mini_ranger.cc:193] Using host: 127.0.191.212
I20260926 06:53:50.295974 767 mini_ranger.cc:227] Ranger admin URL: http://127.0.191.212:35125
Sep 26, 2026 6:53:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 6:53:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 6:53:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 6:53:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 6:53:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 6:53:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 6:53:51 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 06:53:51.412870 767 mini_ranger.cc:160] Time spent starting Ranger: real 8.280s user 0.000s sys 0.009s
Sep 26, 2026 6:54:11 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 6:54:11 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 6:54:11 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 6:54:11 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 6:54:11 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 06:54:12.511978 767 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.191.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.191.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/rangerkms_127.0.191.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/rangerkms_127.0.191.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.191.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.191.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.191.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/HTTP_127.0.191.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.191.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/HTTP_127.0.191.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 06:54:12 dist-test-slave-ws74 krb5kdc[11512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405652, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 06:54:12.595403 767 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 06:54:12.604192 767 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 06:54:13.418845 767 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 06:54:13.418929 767 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-kms
I20260926 06:54:13.418947 767 mini_ranger_kms.cc:227] Using postgres at 127.0.191.212:51059
2026-09-26 06:54:13,618 [I] DB FLAVOR :POSTGRES
2026-09-26 06:54:13,618 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 06:54:13,618 [I] Checking connection
2026-09-26 06:54:13,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 06:54:13,853 [I] connection success
2026-09-26 06:54:13,853 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 06:54:13,853 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 06:54:13,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 06:54:14,081 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 06:54:14,082 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 06:54:14,082 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 06:54:14,082 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:51059/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 06:54:14,323 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 06:54:14.745707 767 mini_ranger_kms.cc:310] Created kms service
I20260926 06:54:14.919476 767 mini_ranger_kms.cc:326] Created kudu user
I20260926 06:54:14.962419 767 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 06:54:14 dist-test-slave-ws74 krb5kdc[11512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405652, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:54:15.314747 767 mini_ranger_kms.cc:384] Added ranger policy
I20260926 06:54:15.314867 767 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-kms:/tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 06:54:15.314888 767 mini_ranger_kms.cc:240] Using host: 127.0.191.212
I20260926 06:54:15.318461 767 mini_ranger_kms.cc:276] Ranger KMS PID: 11905
I20260926 06:54:15.318569 767 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.191.212:53289
Sep 26, 2026 6:54:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 6:54:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 6:54:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 6:54:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 6:54:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 6:54:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 06:54:17.532178 767 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.937s user 0.007s sys 0.174s
I20260926 06:54:17.532289 767 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 06:54:17.532378 767 mini_ranger_kms.cc:401] 127.0.191.212:53289/kms/v1/keys
Sep 26 06:54:17 dist-test-slave-ws74 krb5kdc[11512](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790405657, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.191.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 06:54:18 dist-test-slave-ws74 krb5kdc[11512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405657, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.191.212@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
Sep 26 06:54:18 dist-test-slave-ws74 krb5kdc[11512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405657, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.191.212@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
Sep 26 06:54:20 dist-test-slave-ws74 krb5kdc[11512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:54:20.621479 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:38985
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:38985
--ranger_config_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.191.212:53289/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:54:20.700954 11956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:54:20.701190 11956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:54:20.701224 11956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:54:20.702380 11956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:54:20.702453 11956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:54:20.702477 11956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:54:20.702498 11956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:54:20.702524 11956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:54:20.704030 11956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.191.212:53289/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:38985
--ranger_config_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:38985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:54:20.704293 11956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:54:20.704515 11956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:54:20.710561 11956 server_base.cc:1034] running on GCE node
W20260926 06:54:20.710538 11962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:54:20.710716 11964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:54:20.710546 11961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:54:20.711081 11956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:54:20.711374 11956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:54:20.712551 11956 hybrid_clock.cc:648] HybridClock initialized: now 1790405660712529 us; error 39 us; skew 500 ppm
Sep 26 06:54:20 dist-test-slave-ws74 krb5kdc[11512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405660, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:54:20.715124 11956 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:54:20.715613 11956 webserver.cc:460] Webserver started at http://127.0.191.254:33361/ using document root <none> and password file <none>
I20260926 06:54:20.715828 11956 fs_manager.cc:359] Metadata directory not provided
I20260926 06:54:20.715867 11956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:54:20.716010 11956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:54:20 dist-test-slave-ws74 krb5kdc[11512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405660, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:54:20.759353 11956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "a776ee9f8d904e91874c4fa701c7dfc0"
format_stamp: "Formatted at 2026-09-26 06:54:20 on dist-test-slave-ws74"
server_key: "c412cf7125e99f58555568ba15fc5701"
server_key_iv: "5a055d9bc410c47d76b35e2c9da058de"
server_key_version: "kuduclusterkey@0"
I20260926 06:54:20.759768 11956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "a776ee9f8d904e91874c4fa701c7dfc0"
format_stamp: "Formatted at 2026-09-26 06:54:20 on dist-test-slave-ws74"
server_key: "c412cf7125e99f58555568ba15fc5701"
server_key_iv: "5a055d9bc410c47d76b35e2c9da058de"
server_key_version: "kuduclusterkey@0"
I20260926 06:54:20.761219 11956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:54:20.773670 11971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:54:20.773849 11956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:54:20.773934 11956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "a776ee9f8d904e91874c4fa701c7dfc0"
format_stamp: "Formatted at 2026-09-26 06:54:20 on dist-test-slave-ws74"
server_key: "c412cf7125e99f58555568ba15fc5701"
server_key_iv: "5a055d9bc410c47d76b35e2c9da058de"
server_key_version: "kuduclusterkey@0"
I20260926 06:54:20.774049 11956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:54:20.796733 11956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:54:20.798871 11956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:54:20.799046 11956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:54:20.804112 11956 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:38985
I20260926 06:54:20.804136 12023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:38985 every 8 connection(s)
I20260926 06:54:20.804505 11956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:54:20.805281 12024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:54:20.806643 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 11956
I20260926 06:54:20.806783 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:54:20.806984 767 external_mini_cluster.cc:1428] Setting key ee38e55b0fc3b5727f7f42903fd67d2b
I20260926 06:54:20.808387 12024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0: Bootstrap starting.
I20260926 06:54:20.809175 12024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0: Neither blocks nor log segments found. Creating new log.
I20260926 06:54:20.809512 12024 log.cc:824] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0: Log is configured to *not* fsync() on all Append() calls
I20260926 06:54:20.810843 12024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0: No bootstrap required, opened a new log
Sep 26 06:54:20 dist-test-slave-ws74 krb5kdc[11512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:54:20.813300 12024 raft_consensus.cc:348] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a776ee9f8d904e91874c4fa701c7dfc0" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38985 } }
I20260926 06:54:20.813500 12024 raft_consensus.cc:374] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:54:20.813572 12024 raft_consensus.cc:729] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a776ee9f8d904e91874c4fa701c7dfc0, State: Initialized, Role: FOLLOWER
I20260926 06:54:20.813735 12024 consensus_queue.cc:260] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a776ee9f8d904e91874c4fa701c7dfc0" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38985 } }
I20260926 06:54:20.813814 12024 raft_consensus.cc:388] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:54:20.813843 12024 raft_consensus.cc:482] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:54:20.813907 12024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:54:20.814987 12024 raft_consensus.cc:504] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a776ee9f8d904e91874c4fa701c7dfc0" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38985 } }
I20260926 06:54:20.815091 12024 leader_election.cc:302] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a776ee9f8d904e91874c4fa701c7dfc0; no voters:
I20260926 06:54:20.815250 12024 leader_election.cc:288] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:54:20.815326 12029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:54:20.815550 12029 raft_consensus.cc:686] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [term 1 LEADER]: Becoming Leader. State: Replica: a776ee9f8d904e91874c4fa701c7dfc0, State: Running, Role: LEADER
I20260926 06:54:20.815603 12024 sys_catalog.cc:564] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:54:20.815747 12029 consensus_queue.cc:237] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a776ee9f8d904e91874c4fa701c7dfc0" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38985 } }
I20260926 06:54:20.816104 12029 sys_catalog.cc:455] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a776ee9f8d904e91874c4fa701c7dfc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a776ee9f8d904e91874c4fa701c7dfc0" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38985 } } }
I20260926 06:54:20.816207 12030 sys_catalog.cc:455] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a776ee9f8d904e91874c4fa701c7dfc0. Latest consensus state: current_term: 1 leader_uuid: "a776ee9f8d904e91874c4fa701c7dfc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a776ee9f8d904e91874c4fa701c7dfc0" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 38985 } } }
I20260926 06:54:20.816763 12029 sys_catalog.cc:458] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [sys.catalog]: This master's current role is: LEADER
I20260926 06:54:20.817173 12024 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 06:54:20.817677 12027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:20.807586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53588 (local address 127.0.191.254:38985)
0926 06:54:20.807738 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 06:54:20.807751 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 06:54:20.807774 (+ 23us) server_negotiation.cc:407] Connection header received
0926 06:54:20.808173 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:54:20.808179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:54:20.808224 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:54:20.808378 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:54:20.809028 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:20.810171 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:20.811030 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:20.811461 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:20.811578 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:20.814106 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:54:20.814110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:54:20.814111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:54:20.814129 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:54:20.816071 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:54:20.816483 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:54:20.816486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:54:20.816486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:54:20.816578 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:54:20.816779 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:54:20.816779 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:54:20.816780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:54:20.817063 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:54:20.817235 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 06:54:20.817333 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 06:54:20.817464 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260926 06:54:20.817783 12030 sys_catalog.cc:458] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0 [sys.catalog]: This master's current role is: LEADER
I20260926 06:54:20.820225 12043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:54:20.821281 12043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:54:20.823429 12043 catalog_manager.cc:1347] Generated new cluster ID: 08083ab072d547269a607318edfe69e8
I20260926 06:54:20.823479 12043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:54:20.882516 12043 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:54:20.883387 12043 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:54:20.892185 12043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0: Generated new TSK 0
I20260926 06:54:20.892442 12043 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:54:21.548839 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.191.212:53289/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38985
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:54:21.665531 12069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:54:21.665750 12069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:54:21.665786 12069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:54:21.666940 12069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:54:21.667011 12069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:54:21.667052 12069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:54:21.668467 12069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.191.212:53289/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:54:21.668717 12069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:54:21.668948 12069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:54:21.674283 12075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:54:21.674402 12077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:54:21.674297 12074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:54:21.674777 12069 server_base.cc:1034] running on GCE node
I20260926 06:54:21.674973 12069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:54:21.675251 12069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:54:21.676445 12069 hybrid_clock.cc:648] HybridClock initialized: now 1790405661676436 us; error 28 us; skew 500 ppm
Sep 26 06:54:21 dist-test-slave-ws74 krb5kdc[11512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405661, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:54:21.679121 12069 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:54:21.679545 12069 webserver.cc:460] Webserver started at http://127.0.191.193:33241/ using document root <none> and password file <none>
I20260926 06:54:21.679774 12069 fs_manager.cc:359] Metadata directory not provided
I20260926 06:54:21.679849 12069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:54:21.679981 12069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:54:21 dist-test-slave-ws74 krb5kdc[11512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405661, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:54:21.693605 12069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "17081a64b3ff4c15a8c0d32d6785b3f1"
format_stamp: "Formatted at 2026-09-26 06:54:21 on dist-test-slave-ws74"
server_key: "8914c89fc3803f8486572ff4c57eace6"
server_key_iv: "877dc4b686be4e4a0acb4504fd4ccb98"
server_key_version: "kuduclusterkey@0"
I20260926 06:54:21.694130 12069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "17081a64b3ff4c15a8c0d32d6785b3f1"
format_stamp: "Formatted at 2026-09-26 06:54:21 on dist-test-slave-ws74"
server_key: "8914c89fc3803f8486572ff4c57eace6"
server_key_iv: "877dc4b686be4e4a0acb4504fd4ccb98"
server_key_version: "kuduclusterkey@0"
I20260926 06:54:21.695595 12069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:54:21.708712 12084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:54:21.708940 12069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:54:21.709048 12069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "17081a64b3ff4c15a8c0d32d6785b3f1"
format_stamp: "Formatted at 2026-09-26 06:54:21 on dist-test-slave-ws74"
server_key: "8914c89fc3803f8486572ff4c57eace6"
server_key_iv: "877dc4b686be4e4a0acb4504fd4ccb98"
server_key_version: "kuduclusterkey@0"
I20260926 06:54:21.709177 12069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:54:21.730705 12069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:54:21.733609 12069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:54:21.733752 12069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:54:21.733960 12069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:54:21.734417 12069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:54:21.734520 12069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:54:21.734619 12069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:54:21.734695 12069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:54:21.741732 12069 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:45943
I20260926 06:54:21.742221 12069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:54:21.745702 12199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:45943 every 8 connection(s)
I20260926 06:54:21.746577 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 12069
I20260926 06:54:21.746673 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:54:21.746878 767 external_mini_cluster.cc:1428] Setting key a33ee2b5e9aa15aeac7d05deef5486cc
Sep 26 06:54:21 dist-test-slave-ws74 krb5kdc[11512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405661, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:54:21.759732 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:21.743018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:38985 (local address 127.0.191.193:34623)
0926 06:54:21.743245 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 06:54:21.743294 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 06:54:21.743917 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:54:21.749773 (+ 5856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:54:21.749778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:54:21.750042 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:54:21.750335 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:21.750342 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:21.751423 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:21.751424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:21.752696 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:21.752699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:21.753226 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:21.753227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:21.753367 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:21.754019 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:54:21.754029 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:54:21.755793 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:54:21.757947 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:54:21.757953 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:54:21.757956 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:54:21.758279 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:54:21.758516 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:54:21.758517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:54:21.758517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:54:21.758659 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:54:21.758976 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:54:21.758982 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:54:21.759156 (+ 174us) client_negotiation.cc:769] Sending connection context
0926 06:54:21.759254 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 06:54:21.759418 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":97,"threads_started":1}
I20260926 06:54:21.759915 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:21.743340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:34623 (local address 127.0.191.254:38985)
0926 06:54:21.749528 (+ 6188us) server_negotiation.cc:206] Beginning negotiation
0926 06:54:21.749531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:54:21.749540 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:54:21.749564 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:54:21.749565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:54:21.749579 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:54:21.750436 (+ 857us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:54:21.750477 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:21.751314 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:21.752797 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:21.753135 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:21.754768 (+ 1633us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:21.755932 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:54:21.755934 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:54:21.755934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:54:21.755942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:54:21.757844 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:54:21.758374 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:54:21.758375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:54:21.758376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:54:21.758441 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:54:21.758740 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:54:21.758741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:54:21.758741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:54:21.758894 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:54:21.759649 (+ 755us) server_negotiation.cc:1035] Waiting for connection context
0926 06:54:21.759712 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 06:54:21.759807 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6154,"thread_start_us":75,"threads_started":1}
I20260926 06:54:21.761080 12200 heartbeater.cc:347] Connected to a master server at 127.0.191.254:38985
I20260926 06:54:21.761180 12200 heartbeater.cc:464] Registering TS with master...
I20260926 06:54:21.761368 12200 heartbeater.cc:511] Master 127.0.191.254:38985 requested a full tablet report, sending...
I20260926 06:54:21.762086 11988 ts_manager.cc:194] Registered new tserver with Master: 17081a64b3ff4c15a8c0d32d6785b3f1 (127.0.191.193:45943)
I20260926 06:54:21.763010 11988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:34623
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:54:21.784495 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.191.212:53289/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38985
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:54:21.900556 12206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:54:21.900897 12206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:54:21.900988 12206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:54:21.902773 12206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:54:21.902839 12206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:54:21.902881 12206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:54:21.905051 12206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.191.212:53289/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:54:21.905326 12206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:54:21.905679 12206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Sep 26 06:54:21 dist-test-slave-ws74 krb5kdc[11512](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790405661, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 06:54:21.913151 12214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:54:21.913272 12213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:54:21.913183 12216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:54:21.913537 12206 server_base.cc:1034] running on GCE node
I20260926 06:54:21.913745 12206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:54:21.914036 12206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:54:21.915226 12206 hybrid_clock.cc:648] HybridClock initialized: now 1790405661915202 us; error 37 us; skew 500 ppm
Sep 26 06:54:21 dist-test-slave-ws74 krb5kdc[11512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405661, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:54:21.918856 12206 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:54:21.919373 12206 webserver.cc:460] Webserver started at http://127.0.191.194:32833/ using document root <none> and password file <none>
I20260926 06:54:21.919631 12206 fs_manager.cc:359] Metadata directory not provided
I20260926 06:54:21.919701 12206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:54:21.919852 12206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:54:21 dist-test-slave-ws74 krb5kdc[11512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405661, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:54:21.951905 12206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4"
format_stamp: "Formatted at 2026-09-26 06:54:21 on dist-test-slave-ws74"
server_key: "035673f04b211ede2d3b4f5fb8f0a919"
server_key_iv: "0de45a527c5ef17da93d16c1a5420c75"
server_key_version: "kuduclusterkey@0"
I20260926 06:54:21.952334 12206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4"
format_stamp: "Formatted at 2026-09-26 06:54:21 on dist-test-slave-ws74"
server_key: "035673f04b211ede2d3b4f5fb8f0a919"
server_key_iv: "0de45a527c5ef17da93d16c1a5420c75"
server_key_version: "kuduclusterkey@0"
I20260926 06:54:21.957785 12206 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260926 06:54:21.975298 12223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:54:21.975564 12206 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260926 06:54:21.975661 12206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4"
format_stamp: "Formatted at 2026-09-26 06:54:21 on dist-test-slave-ws74"
server_key: "035673f04b211ede2d3b4f5fb8f0a919"
server_key_iv: "0de45a527c5ef17da93d16c1a5420c75"
server_key_version: "kuduclusterkey@0"
I20260926 06:54:21.975756 12206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:54:21.997654 12206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:54:22.000083 12206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:54:22.000300 12206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:54:22.000563 12206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:54:22.000993 12206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:54:22.001094 12206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:54:22.001168 12206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:54:22.001223 12206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:54:22.007876 12206 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:45051
I20260926 06:54:22.008359 12206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:54:22.010473 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 12206
I20260926 06:54:22.010591 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:54:22.010776 767 external_mini_cluster.cc:1428] Setting key 297c59da610b34f40711657592da8333
Sep 26 06:54:22 dist-test-slave-ws74 krb5kdc[11512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405661, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:54:22.022010 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.009477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:53215 (local address 127.0.191.254:38985)
0926 06:54:22.009527 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:54:22.009530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:54:22.009975 (+ 445us) server_negotiation.cc:407] Connection header received
0926 06:54:22.011386 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:54:22.011388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:54:22.011403 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:54:22.011433 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:54:22.012408 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.013165 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.014692 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.015016 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.015068 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.018486 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:54:22.018489 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:54:22.018489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:54:22.018497 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:54:22.020339 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:54:22.020880 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:54:22.020882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:54:22.020882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:54:22.020949 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:54:22.021215 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:54:22.021216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:54:22.021217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:54:22.021361 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:54:22.021448 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 06:54:22.021792 (+ 344us) server_negotiation.cc:299] Negotiation successful
0926 06:54:22.021896 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:54:22.022889 12340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.009603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:38985 (local address 127.0.191.194:53215)
0926 06:54:22.009854 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0926 06:54:22.009895 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 06:54:22.011263 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:54:22.011724 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:54:22.011729 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:54:22.011976 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:54:22.012289 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.012296 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.013286 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.013286 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.014587 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.014590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.015165 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.015166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.015339 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.016412 (+ 1073us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:54:22.016424 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:54:22.018371 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:54:22.020424 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:54:22.020431 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:54:22.020433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:54:22.020775 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:54:22.021016 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:54:22.021017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:54:22.021017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:54:22.021141 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:54:22.021481 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:54:22.021488 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:54:22.021647 (+ 159us) client_negotiation.cc:769] Sending connection context
0926 06:54:22.022466 (+ 819us) client_negotiation.cc:240] Negotiation successful
0926 06:54:22.022639 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":78,"threads_started":1}
I20260926 06:54:22.023631 12338 heartbeater.cc:347] Connected to a master server at 127.0.191.254:38985
I20260926 06:54:22.023729 12338 heartbeater.cc:464] Registering TS with master...
I20260926 06:54:22.023936 12338 heartbeater.cc:511] Master 127.0.191.254:38985 requested a full tablet report, sending...
I20260926 06:54:22.024370 11988 ts_manager.cc:194] Registered new tserver with Master: b6517ebc561a4eb8b5bd8b90fa77c9e4 (127.0.191.194:45051)
I20260926 06:54:22.024875 11988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:53215
I20260926 06:54:22.025823 12337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:45051 every 8 connection(s)
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:54:22.053122 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.191.212:53289/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38985
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:54:22.184965 12343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:54:22.185412 12343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:54:22.185531 12343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:54:22.187588 12343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:54:22.187704 12343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:54:22.187805 12343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:54:22.190508 12343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.191.212:53289/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:38985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:54:22.191010 12343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:54:22.191321 12343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:54:22.196938 12348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:54:22.197114 12349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:54:22.196938 12351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:54:22.196955 12343 server_base.cc:1034] running on GCE node
I20260926 06:54:22.197508 12343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:54:22.197775 12343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:54:22.198963 12343 hybrid_clock.cc:648] HybridClock initialized: now 1790405662198944 us; error 36 us; skew 500 ppm
Sep 26 06:54:22 dist-test-slave-ws74 krb5kdc[11512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405662, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:54:22.202095 12343 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:54:22.202662 12343 webserver.cc:460] Webserver started at http://127.0.191.195:45303/ using document root <none> and password file <none>
I20260926 06:54:22.202950 12343 fs_manager.cc:359] Metadata directory not provided
I20260926 06:54:22.203006 12343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:54:22.203197 12343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:54:22 dist-test-slave-ws74 krb5kdc[11512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405662, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:54:22.217058 12343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "94e9a411415d427fb559bf628fdfe5c9"
format_stamp: "Formatted at 2026-09-26 06:54:22 on dist-test-slave-ws74"
server_key: "270aa495c66955b6901df1b914e63b8f"
server_key_iv: "80b35f2fc2b363d7d4b5b4505a5b7d84"
server_key_version: "kuduclusterkey@0"
I20260926 06:54:22.217869 12343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "94e9a411415d427fb559bf628fdfe5c9"
format_stamp: "Formatted at 2026-09-26 06:54:22 on dist-test-slave-ws74"
server_key: "270aa495c66955b6901df1b914e63b8f"
server_key_iv: "80b35f2fc2b363d7d4b5b4505a5b7d84"
server_key_version: "kuduclusterkey@0"
I20260926 06:54:22.219822 12343 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260926 06:54:22.237897 12358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:54:22.238158 12343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 06:54:22.238250 12343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "94e9a411415d427fb559bf628fdfe5c9"
format_stamp: "Formatted at 2026-09-26 06:54:22 on dist-test-slave-ws74"
server_key: "270aa495c66955b6901df1b914e63b8f"
server_key_iv: "80b35f2fc2b363d7d4b5b4505a5b7d84"
server_key_version: "kuduclusterkey@0"
I20260926 06:54:22.238339 12343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:54:22.256722 12343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:54:22.259270 12343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:54:22.259476 12343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:54:22.259788 12343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:54:22.260336 12343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:54:22.260439 12343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:54:22.260512 12343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:54:22.260572 12343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:54:22.267838 12343 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:37281
I20260926 06:54:22.268155 12471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:37281 every 8 connection(s)
I20260926 06:54:22.268348 12343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:54:22.273470 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 12343
I20260926 06:54:22.273573 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:54:22.273768 767 external_mini_cluster.cc:1428] Setting key 0d208ebfec437f9cba37db933ecc11a5
Sep 26 06:54:22 dist-test-slave-ws74 krb5kdc[11512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405662, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:54:22.280141 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.269143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:59631 (local address 127.0.191.254:38985)
0926 06:54:22.269192 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:54:22.269194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:54:22.269456 (+ 262us) server_negotiation.cc:407] Connection header received
0926 06:54:22.270167 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:54:22.270169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:54:22.270181 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:54:22.270247 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:54:22.270984 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.271761 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.273274 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.273595 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.273649 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.276721 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:54:22.276722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:54:22.276723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:54:22.276731 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:54:22.278575 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:54:22.279043 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:54:22.279044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:54:22.279044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:54:22.279101 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:54:22.279379 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:54:22.279380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:54:22.279380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:54:22.279524 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:54:22.279567 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 06:54:22.279939 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 06:54:22.280033 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:54:22.280848 12474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.269123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:38985 (local address 127.0.191.195:59631)
0926 06:54:22.269284 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 06:54:22.269340 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 06:54:22.270036 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:54:22.270327 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:54:22.270331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:54:22.270570 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:54:22.270837 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.270843 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.271879 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.271880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.273171 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.273173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.274047 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.274048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.274209 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.274964 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:54:22.274985 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:54:22.276621 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:54:22.278668 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:54:22.278674 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:54:22.278676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:54:22.278955 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:54:22.279183 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:54:22.279184 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:54:22.279184 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:54:22.279310 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:54:22.279614 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:54:22.279620 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:54:22.279785 (+ 165us) client_negotiation.cc:769] Sending connection context
0926 06:54:22.280454 (+ 669us) client_negotiation.cc:240] Negotiation successful
0926 06:54:22.280619 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260926 06:54:22.281489 12472 heartbeater.cc:347] Connected to a master server at 127.0.191.254:38985
I20260926 06:54:22.281574 12472 heartbeater.cc:464] Registering TS with master...
I20260926 06:54:22.281747 12472 heartbeater.cc:511] Master 127.0.191.254:38985 requested a full tablet report, sending...
I20260926 06:54:22.282145 11988 ts_manager.cc:194] Registered new tserver with Master: 94e9a411415d427fb559bf628fdfe5c9 (127.0.191.195:37281)
I20260926 06:54:22.282631 11988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:59631
I20260926 06:54:22.285671 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:54:22.296516 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.287186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53590 (local address 127.0.191.254:38985)
0926 06:54:22.287226 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:54:22.287228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:54:22.288960 (+ 1732us) server_negotiation.cc:407] Connection header received
0926 06:54:22.289034 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:54:22.289034 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:54:22.289045 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:54:22.289096 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:54:22.289873 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.290356 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.291412 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.291605 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.291686 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.293198 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:54:22.293199 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:54:22.293199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:54:22.293207 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:54:22.295029 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:54:22.295431 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:54:22.295433 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:54:22.295433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:54:22.295498 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:54:22.295744 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:54:22.295745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:54:22.295745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:54:22.295893 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:54:22.295996 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 06:54:22.296342 (+ 346us) server_negotiation.cc:299] Negotiation successful
0926 06:54:22.296423 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:54:22.298926 11988 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53590:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:54:22.301306 11988 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:54:22.313414 12484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.307471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:37281 (local address 127.0.0.1:44894)
0926 06:54:22.308856 (+ 1385us) negotiation.cc:107] Waiting for socket to connect
0926 06:54:22.308865 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:54:22.308896 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:54:22.309233 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:54:22.309233 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:54:22.309236 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:54:22.309285 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.309287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.310609 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.310610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.312010 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.312013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.313075 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.313076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.313160 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.313175 (+ 15us) client_negotiation.cc:769] Sending connection context
0926 06:54:22.313240 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 06:54:22.313294 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1371,"thread_start_us":60,"threads_started":1}
I20260926 06:54:22.314014 12486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.307314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:45943 (local address 127.0.0.1:34104)
0926 06:54:22.308261 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0926 06:54:22.308271 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:54:22.308763 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:54:22.309036 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:54:22.309039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:54:22.309042 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:54:22.309662 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.309665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.311233 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.311234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.312713 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.312715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.313775 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.313776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.313850 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.313854 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:54:22.313904 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 06:54:22.313940 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"thread_start_us":72,"threads_started":1}
I20260926 06:54:22.314040 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.307776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44894 (local address 127.0.191.195:37281)
0926 06:54:22.309006 (+ 1230us) server_negotiation.cc:206] Beginning negotiation
0926 06:54:22.309011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:54:22.309020 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:54:22.309059 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:54:22.309061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:54:22.309075 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:54:22.309144 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:54:22.309366 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.310513 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.312122 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.312876 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.312948 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.313895 (+ 947us) server_negotiation.cc:1035] Waiting for connection context
0926 06:54:22.313923 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 06:54:22.313962 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1196,"thread_start_us":83,"threads_started":1}
I20260926 06:54:22.314880 12396 tablet_service.cc:1467] Processing CreateTablet for tablet 77874ef6c43d4e6097fd8333517b7416 (DEFAULT_TABLE table=test-table [id=2d67572d3d3842e0af1849887c61df46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:54:22.315115 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.307750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34104 (local address 127.0.191.193:45943)
0926 06:54:22.308210 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0926 06:54:22.308215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:54:22.308793 (+ 578us) server_negotiation.cc:407] Connection header received
0926 06:54:22.308837 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:54:22.308840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:54:22.308857 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:54:22.308940 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:54:22.310022 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.311135 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.312854 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.313603 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.313684 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.313740 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 06:54:22.314714 (+ 974us) server_negotiation.cc:299] Negotiation successful
0926 06:54:22.314761 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":76,"threads_started":1}
I20260926 06:54:22.315249 12396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77874ef6c43d4e6097fd8333517b7416. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:54:22.315727 12134 tablet_service.cc:1467] Processing CreateTablet for tablet 77874ef6c43d4e6097fd8333517b7416 (DEFAULT_TABLE table=test-table [id=2d67572d3d3842e0af1849887c61df46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:54:22.316038 12134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77874ef6c43d4e6097fd8333517b7416. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:54:22.319801 12491 tablet_bootstrap.cc:492] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1: Bootstrap starting.
I20260926 06:54:22.320998 12491 tablet_bootstrap.cc:654] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1: Neither blocks nor log segments found. Creating new log.
I20260926 06:54:22.321393 12491 log.cc:824] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1: Log is configured to *not* fsync() on all Append() calls
I20260926 06:54:22.322221 12491 tablet_bootstrap.cc:492] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1: No bootstrap required, opened a new log
I20260926 06:54:22.322289 12491 ts_tablet_manager.cc:1397] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 06:54:22.323246 12487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.307059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:45051 (local address 127.0.0.1:58296)
0926 06:54:22.307966 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0926 06:54:22.308010 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:54:22.308078 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:54:22.319240 (+ 11162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:54:22.319242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:54:22.319245 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:54:22.319294 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.319295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.320674 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.320675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.321891 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.321895 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.322983 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.322985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.323053 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.323059 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 06:54:22.323120 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 06:54:22.323168 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"thread_start_us":80,"threads_started":1}
I20260926 06:54:22.323660 12485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.307184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58296 (local address 127.0.191.194:45051)
0926 06:54:22.315385 (+ 8201us) server_negotiation.cc:206] Beginning negotiation
0926 06:54:22.315390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:54:22.315402 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:54:22.315444 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:54:22.315446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:54:22.315462 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:54:22.315548 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:54:22.319429 (+ 3881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.320560 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.322027 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.322879 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.323418 (+ 539us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.323512 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 06:54:22.323540 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 06:54:22.323586 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8159,"thread_start_us":93,"threads_started":1}
I20260926 06:54:22.324088 12490 tablet_bootstrap.cc:492] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9: Bootstrap starting.
I20260926 06:54:22.324329 12272 tablet_service.cc:1467] Processing CreateTablet for tablet 77874ef6c43d4e6097fd8333517b7416 (DEFAULT_TABLE table=test-table [id=2d67572d3d3842e0af1849887c61df46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:54:22.324687 12272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77874ef6c43d4e6097fd8333517b7416. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:54:22.324752 12491 raft_consensus.cc:348] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45051 } } peers { permanent_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45943 } } peers { permanent_uuid: "94e9a411415d427fb559bf628fdfe5c9" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37281 } }
I20260926 06:54:22.324909 12491 raft_consensus.cc:374] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:54:22.324939 12491 raft_consensus.cc:729] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17081a64b3ff4c15a8c0d32d6785b3f1, State: Initialized, Role: FOLLOWER
I20260926 06:54:22.325590 12491 consensus_queue.cc:260] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45051 } } peers { permanent_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45943 } } peers { permanent_uuid: "94e9a411415d427fb559bf628fdfe5c9" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37281 } }
I20260926 06:54:22.325989 12491 ts_tablet_manager.cc:1428] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260926 06:54:22.326143 12200 heartbeater.cc:503] Master 127.0.191.254:38985 was elected leader, sending a full tablet report...
I20260926 06:54:22.327100 12490 tablet_bootstrap.cc:654] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9: Neither blocks nor log segments found. Creating new log.
I20260926 06:54:22.327457 12490 log.cc:824] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9: Log is configured to *not* fsync() on all Append() calls
I20260926 06:54:22.328565 12495 tablet_bootstrap.cc:492] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4: Bootstrap starting.
I20260926 06:54:22.328648 12490 tablet_bootstrap.cc:492] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9: No bootstrap required, opened a new log
I20260926 06:54:22.328778 12490 ts_tablet_manager.cc:1397] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260926 06:54:22.329560 12495 tablet_bootstrap.cc:654] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4: Neither blocks nor log segments found. Creating new log.
I20260926 06:54:22.329932 12495 log.cc:824] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4: Log is configured to *not* fsync() on all Append() calls
I20260926 06:54:22.331141 12495 tablet_bootstrap.cc:492] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4: No bootstrap required, opened a new log
I20260926 06:54:22.331036 12490 raft_consensus.cc:348] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45051 } } peers { permanent_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45943 } } peers { permanent_uuid: "94e9a411415d427fb559bf628fdfe5c9" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37281 } }
I20260926 06:54:22.331184 12490 raft_consensus.cc:374] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:54:22.331205 12495 ts_tablet_manager.cc:1397] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 06:54:22.331215 12490 raft_consensus.cc:729] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94e9a411415d427fb559bf628fdfe5c9, State: Initialized, Role: FOLLOWER
I20260926 06:54:22.331343 12490 consensus_queue.cc:260] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45051 } } peers { permanent_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45943 } } peers { permanent_uuid: "94e9a411415d427fb559bf628fdfe5c9" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37281 } }
I20260926 06:54:22.331553 12490 ts_tablet_manager.cc:1428] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 06:54:22.331660 12472 heartbeater.cc:503] Master 127.0.191.254:38985 was elected leader, sending a full tablet report...
I20260926 06:54:22.333082 12495 raft_consensus.cc:348] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45051 } } peers { permanent_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45943 } } peers { permanent_uuid: "94e9a411415d427fb559bf628fdfe5c9" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37281 } }
I20260926 06:54:22.333240 12495 raft_consensus.cc:374] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:54:22.333279 12495 raft_consensus.cc:729] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6517ebc561a4eb8b5bd8b90fa77c9e4, State: Initialized, Role: FOLLOWER
I20260926 06:54:22.333573 12495 consensus_queue.cc:260] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45051 } } peers { permanent_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45943 } } peers { permanent_uuid: "94e9a411415d427fb559bf628fdfe5c9" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37281 } }
I20260926 06:54:22.333834 12495 ts_tablet_manager.cc:1428] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 06:54:22.333956 12338 heartbeater.cc:503] Master 127.0.191.254:38985 was elected leader, sending a full tablet report...
I20260926 06:54:22.340339 12493 raft_consensus.cc:482] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:54:22.340461 12493 raft_consensus.cc:504] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45051 } } peers { permanent_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45943 } } peers { permanent_uuid: "94e9a411415d427fb559bf628fdfe5c9" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37281 } }
I20260926 06:54:22.340775 12493 leader_election.cc:288] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6517ebc561a4eb8b5bd8b90fa77c9e4 (127.0.191.194:45051), 94e9a411415d427fb559bf628fdfe5c9 (127.0.191.195:37281)
I20260926 06:54:22.345225 12485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.341199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:49021 (local address 127.0.191.194:45051)
0926 06:54:22.341242 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:54:22.341244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:54:22.341526 (+ 282us) server_negotiation.cc:407] Connection header received
0926 06:54:22.341629 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:54:22.341630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:54:22.341640 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:54:22.341670 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:54:22.341964 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.342708 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.344220 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.344769 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.344830 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.344840 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 06:54:22.345118 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 06:54:22.345159 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:54:22.345528 12500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.340940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:45051 (local address 127.0.191.193:49021)
0926 06:54:22.341457 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0926 06:54:22.341470 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:54:22.341554 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:54:22.341734 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:54:22.341735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:54:22.341747 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:54:22.341837 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.341839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.342839 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.342840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.344113 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.344116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.344956 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.344957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.345028 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.345033 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:54:22.345367 (+ 334us) client_negotiation.cc:240] Negotiation successful
0926 06:54:22.345446 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":74,"threads_started":1}
I20260926 06:54:22.345849 12292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77874ef6c43d4e6097fd8333517b7416" candidate_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4" is_pre_election: true
I20260926 06:54:22.345984 12292 raft_consensus.cc:2454] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17081a64b3ff4c15a8c0d32d6785b3f1 in term 0.
I20260926 06:54:22.346197 12087 leader_election.cc:302] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17081a64b3ff4c15a8c0d32d6785b3f1, b6517ebc561a4eb8b5bd8b90fa77c9e4; no voters:
I20260926 06:54:22.346333 12493 raft_consensus.cc:2781] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:54:22.346381 12493 raft_consensus.cc:482] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:54:22.346410 12493 raft_consensus.cc:3037] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:54:22.347277 12493 raft_consensus.cc:504] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45051 } } peers { permanent_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45943 } } peers { permanent_uuid: "94e9a411415d427fb559bf628fdfe5c9" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37281 } }
I20260926 06:54:22.347414 12493 leader_election.cc:288] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [CANDIDATE]: Term 1 election: Requested vote from peers b6517ebc561a4eb8b5bd8b90fa77c9e4 (127.0.191.194:45051), 94e9a411415d427fb559bf628fdfe5c9 (127.0.191.195:37281)
I20260926 06:54:22.347572 12292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77874ef6c43d4e6097fd8333517b7416" candidate_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4"
I20260926 06:54:22.347649 12292 raft_consensus.cc:3037] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:54:22.348428 12292 raft_consensus.cc:2454] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17081a64b3ff4c15a8c0d32d6785b3f1 in term 1.
I20260926 06:54:22.348628 12087 leader_election.cc:302] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17081a64b3ff4c15a8c0d32d6785b3f1, b6517ebc561a4eb8b5bd8b90fa77c9e4; no voters:
I20260926 06:54:22.348706 12493 raft_consensus.cc:2781] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:54:22.348881 12493 raft_consensus.cc:686] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [term 1 LEADER]: Becoming Leader. State: Replica: 17081a64b3ff4c15a8c0d32d6785b3f1, State: Running, Role: LEADER
I20260926 06:54:22.348974 12493 consensus_queue.cc:237] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45051 } } peers { permanent_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45943 } } peers { permanent_uuid: "94e9a411415d427fb559bf628fdfe5c9" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37281 } }
I20260926 06:54:22.349835 11988 catalog_manager.cc:5495] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17081a64b3ff4c15a8c0d32d6785b3f1 (127.0.191.193). New cstate: current_term: 1 leader_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 45943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94e9a411415d427fb559bf628fdfe5c9" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37281 } health_report { overall_health: UNKNOWN } } }
I20260926 06:54:22.351802 12500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.340988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:37281 (local address 127.0.191.193:40629)
0926 06:54:22.345763 (+ 4775us) negotiation.cc:107] Waiting for socket to connect
0926 06:54:22.345773 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:54:22.345850 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:54:22.346021 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:54:22.346022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:54:22.346023 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:54:22.346051 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.347578 (+ 1527us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.348524 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.348525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.350803 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:54:22.350806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.351561 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.351562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:54:22.351627 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.351631 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:54:22.351696 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 06:54:22.351734 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4763,"thread_start_us":94,"threads_started":1}
I20260926 06:54:22.351818 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.341216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:40629 (local address 127.0.191.195:37281)
0926 06:54:22.341274 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 06:54:22.341276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:54:22.345824 (+ 4548us) server_negotiation.cc:407] Connection header received
0926 06:54:22.345923 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:54:22.345924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:54:22.345933 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:54:22.345962 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:54:22.347675 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.348406 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.350901 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.351469 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.351544 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.351552 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:54:22.351712 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 06:54:22.351753 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 06:54:22.352506 12426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77874ef6c43d4e6097fd8333517b7416" candidate_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94e9a411415d427fb559bf628fdfe5c9" is_pre_election: true
I20260926 06:54:22.352514 12425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77874ef6c43d4e6097fd8333517b7416" candidate_uuid: "17081a64b3ff4c15a8c0d32d6785b3f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94e9a411415d427fb559bf628fdfe5c9"
I20260926 06:54:22.352659 12426 raft_consensus.cc:2454] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17081a64b3ff4c15a8c0d32d6785b3f1 in term 0.
I20260926 06:54:22.360051 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.357075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34108 (local address 127.0.191.193:45943)
0926 06:54:22.357115 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:54:22.357117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:54:22.357126 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:54:22.357187 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:54:22.357188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:54:22.357206 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:54:22.357278 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:54:22.357418 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.358134 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.359198 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:54:22.359488 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:54:22.359555 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:54:22.359728 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:54:22.359844 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:54:22.359922 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 06:54:22.359963 (+ 41us) server_negotiation.cc:299] Negotiation successful
0926 06:54:22.359984 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:54:22.361990 12426 raft_consensus.cc:3037] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:54:22.362072 12292 raft_consensus.cc:1261] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4 [term 1 FOLLOWER]: Refusing update from remote peer 17081a64b3ff4c15a8c0d32d6785b3f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:54:22.362396 12501 consensus_queue.cc:1035] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6517ebc561a4eb8b5bd8b90fa77c9e4" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 45051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:54:22.363338 12426 raft_consensus.cc:1261] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9 [term 1 FOLLOWER]: Refusing update from remote peer 17081a64b3ff4c15a8c0d32d6785b3f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:54:22.364076 12501 consensus_queue.cc:1035] T 77874ef6c43d4e6097fd8333517b7416 P 17081a64b3ff4c15a8c0d32d6785b3f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94e9a411415d427fb559bf628fdfe5c9" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:54:22.368564 12505 mvcc.cc:205] Tried to move back new op lower bound from 7333501593032310784 to 7333501592981946368. Current Snapshot: MvccSnapshot[applied={T|T < 7333501593032310784}]
I20260926 06:54:22.369681 11988 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:54:22.369767 11988 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:54:22.370193 12507 mvcc.cc:205] Tried to move back new op lower bound from 7333501593032310784 to 7333501592981946368. Current Snapshot: MvccSnapshot[applied={T|T < 7333501593032310784}]
I20260926 06:54:22.370603 11988 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a776ee9f8d904e91874c4fa701c7dfc0: Sending DeleteTablet for 3 replicas of tablet 77874ef6c43d4e6097fd8333517b7416
I20260926 06:54:22.371150 12272 tablet_service.cc:1514] Processing DeleteTablet for tablet 77874ef6c43d4e6097fd8333517b7416 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:54:22 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:58296
I20260926 06:54:22.371345 12134 tablet_service.cc:1514] Processing DeleteTablet for tablet 77874ef6c43d4e6097fd8333517b7416 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:54:22 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:34104
I20260926 06:54:22.371551 12396 tablet_service.cc:1514] Processing DeleteTablet for tablet 77874ef6c43d4e6097fd8333517b7416 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:54:22 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:44894
I20260926 06:54:22.372201 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 12069
I20260926 06:54:22.376639 12515 tablet_replica.cc:330] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9: stopping tablet replica
I20260926 06:54:22.376787 12513 tablet_replica.cc:330] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4: stopping tablet replica
I20260926 06:54:22.377067 12515 raft_consensus.cc:2229] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:54:22.377203 12513 raft_consensus.cc:2229] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:54:22.377851 12513 raft_consensus.cc:2258] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:54:22.378034 12515 raft_consensus.cc:2258] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:54:22.378774 12513 ts_tablet_manager.cc:1905] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:54:22.379181 12515 ts_tablet_manager.cc:1905] T 77874ef6c43d4e6097fd8333517b7416 P 94e9a411415d427fb559bf628fdfe5c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:54:22.382439 12513 ts_tablet_manager.cc:1918] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:54:22.384457 12513 log.cc:1196] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/77874ef6c43d4e6097fd8333517b7416
I20260926 06:54:22.385175 12513 ts_tablet_manager.cc:1939] T 77874ef6c43d4e6097fd8333517b7416 P b6517ebc561a4eb8b5bd8b90fa77c9e4: Deleting consensus metadata
W20260926 06:54:22.385710 11973 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:45943 (error 108)
I20260926 06:54:22.386374 11974 catalog_manager.cc:4841] TS b6517ebc561a4eb8b5bd8b90fa77c9e4 (127.0.191.194:45051): tablet 77874ef6c43d4e6097fd8333517b7416 (table test-table [id=2d67572d3d3842e0af1849887c61df46]) successfully deleted
I20260926 06:54:22.387003 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 12206
I20260926 06:54:22.388275 12487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:54:22.388162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:45943 (local address 127.0.0.1:34114)
0926 06:54:22.388209 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 06:54:22.388239 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 06:54:22.388468 11973 catalog_manager.cc:4584] TS 17081a64b3ff4c15a8c0d32d6785b3f1 (127.0.191.193:45943): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77874ef6c43d4e6097fd8333517b7416: Network error: Client connection negotiation failed: client connection to 127.0.191.193:45943: connect: Connection refused (error 111)
I20260926 06:54:22.392191 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 12343
W20260926 06:54:22.400115 12487 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:54:22.399943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:37281 (local address 127.0.0.1:44900)
0926 06:54:22.400020 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0926 06:54:22.400048 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.195:37281: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 06:54:22.400342 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 11956
I20260926 06:54:22.405798 767 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 06:54:22.751157 767 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 06:54:22.751258 767 mini_ranger.cc:66] Stopping Ranger...
I20260926 06:54:23.095534 767 mini_ranger.cc:68] Stopped Ranger
2026-09-26 06:54:23.095 UTC [11542] LOG: received smart shutdown request
2026-09-26 06:54:23.099 UTC [11542] LOG: background worker "logical replication launcher" (PID 11551) exited with exit code 1
2026-09-26 06:54:23.099 UTC [11546] LOG: shutting down
2026-09-26 06:54:23.279 UTC [11542] LOG: database system is shut down
2026-09-26T06:54:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44889 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:54:23 dist-test-slave-ws74 krb5kdc[12525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:54:23 dist-test-slave-ws74 krb5kdc[12525](info): set up 2 sockets
Sep 26 06:54:23 dist-test-slave-ws74 krb5kdc[12525](info): commencing operation
krb5kdc: starting...
W20260926 06:54:24.433285 767 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:54:24 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:54:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:54:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.191.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.191.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/rangeradmin_127.0.191.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/rangeradmin_127.0.191.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.191.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.191.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/rangerlookup_127.0.191.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/rangerlookup_127.0.191.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.191.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.191.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/HTTP_127.0.191.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/HTTP_127.0.191.212@KRBTEST.COM.keytab.
I20260926 06:54:24.594280 767 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskyAhorJ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/postgres -l logfile start
2026-09-26 06:54:27.107 UTC [12555] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 06:54:27.107 UTC [12555] LOG: listening on IPv4 address "127.0.191.212", port 54581
2026-09-26 06:54:27.110 UTC [12555] LOG: listening on Unix socket "/tmp/.s.PGSQL.54581"
2026-09-26 06:54:27.131 UTC [12558] LOG: database system was shut down at 2026-09-26 06:54:25 UTC
2026-09-26 06:54:27.135 UTC [12555] LOG: database system is ready to accept connections
I20260926 06:54:28.089244 767 mini_postgres.cc:96] Postgres bound to 54581
2026-09-26 06:54:28.095 UTC [12566] FATAL: database "slave" does not exist
127.0.191.212:54581 - accepting connections
I20260926 06:54:28.095714 767 mini_ranger.cc:161] Starting Ranger...
I20260926 06:54:28.110175 767 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 06:54:28.611330 767 mini_ranger.cc:87] Created ranger Postgres database
I20260926 06:54:28.611418 767 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-admin
2026-09-26 06:54:28,891 [I] DB FLAVOR :POSTGRES
2026-09-26 06:54:28,892 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 06:54:28,892 [I] Checking connection..
2026-09-26 06:54:28,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 06:54:29,177 [I] Checking connection passed.
2026-09-26 06:54:29,177 [I] --------- Verifying version history table ---------
2026-09-26 06:54:29,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 06:54:29,404 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 06:54:29,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 06:54:29,654 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 06:54:29,655 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 06:54:29,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 06:54:29.902 UTC [12630] WARNING: there is no transaction in progress
2026-09-26 06:54:29,908 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 06:54:29,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 06:54:30,160 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 06:54:30,160 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 06:54:30,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 06:54:30,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 06:54:30,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ws74.c.gcp-upstream.internal','N') ;"
2026-09-26 06:54:30,858 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 06:54:31,117 [I] Table x_portal_user does not exist in database ranger
2026-09-26 06:54:31,118 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 06:54:31,373 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 06:54:31,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 06:54:31,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 06:54:31,842 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 06:54:31,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 06:54:32.786 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:32.801 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:32.818 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:32.833 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:32.849 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:32.866 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:32.929 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:32.935 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:32.950 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:32.959 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:32.969 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:32.982 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:32.992 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:32.998 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:33.006 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:33.013 UTC [12738] WARNING: there is no transaction in progress
2026-09-26 06:54:33,664 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 06:54:33,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 06:54:33,906 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 06:54:33,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 06:54:34,157 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 06:54:34,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 06:54:34,389 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 06:54:34,389 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 06:54:34,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 06:54:34,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ws74.c.gcp-upstream.internal';"
2026-09-26 06:54:35,140 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 06:54:35,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 06:54:35,382 [I] DB_PATCHES have already been applied
I20260926 06:54:35.388073 767 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-admin:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskyAhorJ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 06:54:35.388139 767 mini_ranger.cc:193] Using host: 127.0.191.212
I20260926 06:54:35.391242 767 mini_ranger.cc:227] Ranger admin URL: http://127.0.191.212:39363
Sep 26, 2026 6:54:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 6:54:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 6:54:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 6:54:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 6:54:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 6:54:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 6:54:36 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 06:54:36.555065 767 mini_ranger.cc:160] Time spent starting Ranger: real 8.459s user 0.002s sys 0.006s
Sep 26, 2026 6:54:55 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 6:54:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 6:54:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 6:54:55 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 6:54:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 6:54:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 6:54:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 6:54:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 6:54:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 6:54:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 6:54:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 6:54:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 6:54:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 6:54:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 6:54:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 6:54:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 6:54:55 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 6:54:56 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 06:54:56.871805 767 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.191.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.191.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/rangerkms_127.0.191.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.191.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/rangerkms_127.0.191.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.191.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.191.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.191.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/HTTP_127.0.191.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.191.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/HTTP_127.0.191.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 06:54:56 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405696, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 06:54:56.972258 767 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 06:54:56.982023 767 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 06:54:57.824474 767 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 06:54:57.824555 767 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-kms
I20260926 06:54:57.824576 767 mini_ranger_kms.cc:227] Using postgres at 127.0.191.212:54581
2026-09-26 06:54:57,998 [I] DB FLAVOR :POSTGRES
2026-09-26 06:54:57,998 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 06:54:57,998 [I] Checking connection
2026-09-26 06:54:57,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 06:54:58,245 [I] connection success
2026-09-26 06:54:58,245 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 06:54:58,245 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 06:54:58,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 06:54:58,486 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 06:54:58,487 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 06:54:58,487 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 06:54:58,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.191.212:54581/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 06:54:58,745 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 06:54:59.194461 767 mini_ranger_kms.cc:310] Created kms service
I20260926 06:54:59.358903 767 mini_ranger_kms.cc:326] Created kudu user
I20260926 06:54:59.401567 767 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 06:54:59 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405696, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:54:59.769765 767 mini_ranger_kms.cc:384] Added ranger policy
I20260926 06:54:59.769872 767 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-kms:/tmp/dist-test-taskyAhorJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskyAhorJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 06:54:59.769891 767 mini_ranger_kms.cc:240] Using host: 127.0.191.212
I20260926 06:54:59.773491 767 mini_ranger_kms.cc:276] Ranger KMS PID: 12918
I20260926 06:54:59.773567 767 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.191.212:46143
Sep 26, 2026 6:55:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 6:55:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 6:55:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 6:55:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 6:55:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 6:55:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 06:55:02.000085 767 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.028s user 0.000s sys 0.156s
I20260926 06:55:02.000190 767 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 06:55:02.000240 767 mini_ranger_kms.cc:401] 127.0.191.212:46143/kms/v1/keys
Sep 26 06:55:02 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790405702, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.191.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 06:55:02 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405702, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.191.212@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
Sep 26 06:55:02 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405702, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.191.212@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
Sep 26 06:55:04 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:55:05.033162 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:45911
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:45911
--ranger_config_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.191.212:46143/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:05.124199 12969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:05.124402 12969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:05.124423 12969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:05.125627 12969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:55:05.125666 12969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:05.125680 12969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:05.125699 12969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:55:05.125727 12969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:55:05.127278 12969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.191.212:46143/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:45911
--ranger_config_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:45911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:05.127547 12969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:05.127797 12969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:05.133932 12974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:05.134053 12975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:05.133953 12969 server_base.cc:1034] running on GCE node
W20260926 06:55:05.134158 12977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:05.134645 12969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:05.134943 12969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:05.136107 12969 hybrid_clock.cc:648] HybridClock initialized: now 1790405705136086 us; error 40 us; skew 500 ppm
Sep 26 06:55:05 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405705, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:05.143680 12969 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:55:05.144193 12969 webserver.cc:460] Webserver started at http://127.0.191.254:34091/ using document root <none> and password file <none>
I20260926 06:55:05.144446 12969 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:05.144503 12969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:05.144645 12969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:55:05 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405705, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:55:05.204414 12969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "8ec8fd56f99744abb2d9de674930e6d2"
format_stamp: "Formatted at 2026-09-26 06:55:05 on dist-test-slave-ws74"
server_key: "bb34bdc01c88dae8edd1e3e5852cb507"
server_key_iv: "4eb5a2c8ee3917bc8911dba50cf568a1"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:05.204895 12969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "8ec8fd56f99744abb2d9de674930e6d2"
format_stamp: "Formatted at 2026-09-26 06:55:05 on dist-test-slave-ws74"
server_key: "bb34bdc01c88dae8edd1e3e5852cb507"
server_key_iv: "4eb5a2c8ee3917bc8911dba50cf568a1"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:05.206689 12969 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260926 06:55:05.220209 12984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:05.220425 12969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:55:05.220516 12969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "8ec8fd56f99744abb2d9de674930e6d2"
format_stamp: "Formatted at 2026-09-26 06:55:05 on dist-test-slave-ws74"
server_key: "bb34bdc01c88dae8edd1e3e5852cb507"
server_key_iv: "4eb5a2c8ee3917bc8911dba50cf568a1"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:05.220603 12969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:05.238291 12969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:05.239967 12969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:05.240125 12969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:05.244535 12969 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:45911
I20260926 06:55:05.244580 13036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:45911 every 8 connection(s)
I20260926 06:55:05.244947 12969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:55:05.245599 13037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:05.247825 13037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2: Bootstrap starting.
I20260926 06:55:05.248415 13037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:05.248706 13037 log.cc:824] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:05.249428 13037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2: No bootstrap required, opened a new log
I20260926 06:55:05.249439 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 12969
I20260926 06:55:05.249554 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:55:05.249739 767 external_mini_cluster.cc:1428] Setting key 911e97ea36a2f0c2c7fbc9cfaf069f2d
I20260926 06:55:05.251688 13037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec8fd56f99744abb2d9de674930e6d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45911 } }
I20260926 06:55:05.251844 13037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:05.251907 13037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ec8fd56f99744abb2d9de674930e6d2, State: Initialized, Role: FOLLOWER
I20260926 06:55:05.252053 13037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec8fd56f99744abb2d9de674930e6d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45911 } }
I20260926 06:55:05.252142 13037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:55:05.252182 13037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:55:05.252243 13037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:05.252774 13037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec8fd56f99744abb2d9de674930e6d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45911 } }
I20260926 06:55:05.252889 13037 leader_election.cc:302] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ec8fd56f99744abb2d9de674930e6d2; no voters:
I20260926 06:55:05.253087 13037 leader_election.cc:288] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:55:05.253126 13042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:55:05.253242 13042 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 1 LEADER]: Becoming Leader. State: Replica: 8ec8fd56f99744abb2d9de674930e6d2, State: Running, Role: LEADER
I20260926 06:55:05.253352 13037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:55:05.253410 13042 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec8fd56f99744abb2d9de674930e6d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45911 } }
I20260926 06:55:05.253863 13043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ec8fd56f99744abb2d9de674930e6d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec8fd56f99744abb2d9de674930e6d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45911 } } }
I20260926 06:55:05.253948 13043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:05.254158 13044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ec8fd56f99744abb2d9de674930e6d2. Latest consensus state: current_term: 1 leader_uuid: "8ec8fd56f99744abb2d9de674930e6d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec8fd56f99744abb2d9de674930e6d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45911 } } }
I20260926 06:55:05.254439 13044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:05.254609 13049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:55:05.254917 13037 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 06:55:05.255131 13049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 06:55:05 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:05.260720 13049 catalog_manager.cc:1347] Generated new cluster ID: 805d1afc9a2940b18d9315ba9d2516f5
I20260926 06:55:05.260848 13049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:55:05.265957 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:05.250232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.0.191.254:45911)
0926 06:55:05.250402 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:05.250413 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:05.250447 (+ 34us) server_negotiation.cc:407] Connection header received
0926 06:55:05.251683 (+ 1236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:05.251689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:05.251735 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:05.251874 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:05.252618 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:05.253826 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:05.254920 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:05.257620 (+ 2700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:05.258197 (+ 577us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:05.261519 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:05.261523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:05.261524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:05.261535 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:05.263813 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:05.264274 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:05.264277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:05.264277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:05.264396 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:05.264627 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:05.264628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:05.264628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:05.264974 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:05.265502 (+ 528us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:05.265619 (+ 117us) server_negotiation.cc:299] Negotiation successful
0926 06:55:05.265745 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260926 06:55:05.277999 13049 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:55:05.278733 13049 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:55:05.281328 13049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2: Generated new TSK 0
I20260926 06:55:05.281546 13049 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:55:05.956243 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.191.212:46143/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:45911
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:06.049866 13082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:06.050110 13082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:06.050161 13082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:06.051453 13082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:06.051522 13082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:06.051565 13082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:55:06.053007 13082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.191.212:46143/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:06.053308 13082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:06.053561 13082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:06.058770 13087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:06.058772 13088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:06.058792 13090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:06.059259 13082 server_base.cc:1034] running on GCE node
I20260926 06:55:06.059477 13082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:06.059756 13082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:06.060940 13082 hybrid_clock.cc:648] HybridClock initialized: now 1790405706060921 us; error 24 us; skew 500 ppm
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:06.063921 13082 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:55:06.064414 13082 webserver.cc:460] Webserver started at http://127.0.191.193:35031/ using document root <none> and password file <none>
I20260926 06:55:06.064638 13082 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:06.064703 13082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:06.064857 13082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:55:06.078388 13082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d"
format_stamp: "Formatted at 2026-09-26 06:55:06 on dist-test-slave-ws74"
server_key: "f415dd420839480c4eb860c7a2c22bcd"
server_key_iv: "5cedc64b1c1d515f46d554923b47d04d"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:06.078943 13082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d"
format_stamp: "Formatted at 2026-09-26 06:55:06 on dist-test-slave-ws74"
server_key: "f415dd420839480c4eb860c7a2c22bcd"
server_key_iv: "5cedc64b1c1d515f46d554923b47d04d"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:06.081759 13082 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260926 06:55:06.095080 13097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:06.095584 13082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:06.095680 13082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d"
format_stamp: "Formatted at 2026-09-26 06:55:06 on dist-test-slave-ws74"
server_key: "f415dd420839480c4eb860c7a2c22bcd"
server_key_iv: "5cedc64b1c1d515f46d554923b47d04d"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:06.095767 13082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:06.118758 13082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:06.121306 13082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:06.121551 13082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:06.121904 13082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:06.122344 13082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:06.122442 13082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:06.122545 13082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:06.122617 13082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:06.129668 13082 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:33311
I20260926 06:55:06.130160 13082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:55:06.131661 13212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:33311 every 8 connection(s)
I20260926 06:55:06.131942 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 13082
I20260926 06:55:06.132052 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:55:06.132267 767 external_mini_cluster.cc:1428] Setting key de3ff7682213622664924aed88e801e7
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:06.145907 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:06.134541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:34381 (local address 127.0.191.254:45911)
0926 06:55:06.135587 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:06.135590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:06.135597 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:55:06.135620 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:06.135621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:06.135633 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:06.135692 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:06.136466 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:06.137287 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:06.138805 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:06.139108 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:06.139172 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:06.141689 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:06.141691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:06.141691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:06.141698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:06.144271 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:06.144757 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:06.144759 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:06.144759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:06.144823 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:06.145104 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:06.145105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:06.145105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:06.145249 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:06.145307 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:06.145712 (+ 405us) server_negotiation.cc:299] Negotiation successful
0926 06:55:06.145806 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1019,"thread_start_us":98,"threads_started":1}
I20260926 06:55:06.146050 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:06.132568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:45911 (local address 127.0.191.193:34381)
0926 06:55:06.134747 (+ 2179us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:06.134795 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:06.135410 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:06.135815 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:06.135819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:06.136072 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:06.136357 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:06.136364 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:06.137411 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:06.137412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:06.138675 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:06.138677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:06.139261 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:06.139262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:06.139402 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:06.140034 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:06.140045 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:06.141589 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:06.144366 (+ 2777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:06.144373 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:06.144375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:06.144668 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:06.144901 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:06.144902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:06.144902 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:06.145032 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:06.145334 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:06.145339 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:06.145541 (+ 202us) client_negotiation.cc:769] Sending connection context
0926 06:55:06.145652 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 06:55:06.145810 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2130,"thread_start_us":78,"threads_started":1}
I20260926 06:55:06.146584 13213 heartbeater.cc:347] Connected to a master server at 127.0.191.254:45911
I20260926 06:55:06.146679 13213 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:06.146867 13213 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
I20260926 06:55:06.147398 13001 ts_manager.cc:194] Registered new tserver with Master: 4cc3f6ecd1b140e3a354fec8fcb9c80d (127.0.191.193:33311)
I20260926 06:55:06.148273 13001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:34381
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:55:06.168870 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.191.212:46143/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:45911
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:06.285243 13219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:06.285605 13219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:06.285691 13219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:06.287477 13219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:06.287603 13219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:06.287688 13219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:55:06.289944 13219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.191.212:46143/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:06.290403 13219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:06.290741 13219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:06.297407 13227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:06.297487 13226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:06.297811 13229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:06.298622 13219 server_base.cc:1034] running on GCE node
I20260926 06:55:06.298926 13219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:06.299186 13219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:06.300349 13219 hybrid_clock.cc:648] HybridClock initialized: now 1790405706300330 us; error 33 us; skew 500 ppm
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:06.303603 13219 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:55:06.304090 13219 webserver.cc:460] Webserver started at http://127.0.191.194:46491/ using document root <none> and password file <none>
I20260926 06:55:06.304332 13219 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:06.304382 13219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:06.304503 13219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:55:06.323673 13219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "91e1bb2caa5040b59d7d092cf7a0f600"
format_stamp: "Formatted at 2026-09-26 06:55:06 on dist-test-slave-ws74"
server_key: "2c3b43a291fc55b03c15841a9ab9fa87"
server_key_iv: "ae3478efac000faeb52da4dc9b5a503d"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:06.325129 13219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "91e1bb2caa5040b59d7d092cf7a0f600"
format_stamp: "Formatted at 2026-09-26 06:55:06 on dist-test-slave-ws74"
server_key: "2c3b43a291fc55b03c15841a9ab9fa87"
server_key_iv: "ae3478efac000faeb52da4dc9b5a503d"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:06.327303 13219 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:06.345088 13236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:06.345319 13219 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:55:06.345436 13219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "91e1bb2caa5040b59d7d092cf7a0f600"
format_stamp: "Formatted at 2026-09-26 06:55:06 on dist-test-slave-ws74"
server_key: "2c3b43a291fc55b03c15841a9ab9fa87"
server_key_iv: "ae3478efac000faeb52da4dc9b5a503d"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:06.345530 13219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:06.374145 13219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:06.376513 13219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:06.376696 13219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:06.376946 13219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:06.377346 13219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:06.377456 13219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:06.377523 13219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:06.377578 13219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:06.383798 13219 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:43135
I20260926 06:55:06.384255 13219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:55:06.385344 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 13219
I20260926 06:55:06.385481 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:55:06.385730 767 external_mini_cluster.cc:1428] Setting key 06116988bbd67f9a163fae30b093d0ad
I20260926 06:55:06.386955 13349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:43135 every 8 connection(s)
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:06.401240 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:06.388647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:56539 (local address 127.0.191.254:45911)
0926 06:55:06.388723 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:06.388738 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:06.389079 (+ 341us) server_negotiation.cc:407] Connection header received
0926 06:55:06.389836 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:06.389837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:06.389851 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:06.389892 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:06.390664 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:06.391421 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:06.392838 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:06.393144 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:06.393184 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:06.397953 (+ 4769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:06.397955 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:06.397955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:06.397963 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:06.399740 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:06.400232 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:06.400233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:06.400233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:06.400288 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:06.400544 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:06.400545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:06.400545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:06.400669 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:06.400709 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:06.401059 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 06:55:06.401143 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 06:55:06.401966 13353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:06.388382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:45911 (local address 127.0.191.194:56539)
0926 06:55:06.388958 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:06.389003 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:06.389718 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:06.390071 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:06.390075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:06.390301 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:06.390562 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:06.390568 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:06.391532 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:06.391533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:06.392738 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:06.392741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:06.393280 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:06.393281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:06.393440 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:06.396142 (+ 2702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:06.396152 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:06.397829 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:06.399859 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:06.399865 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:06.399877 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:06.400146 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:06.400355 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:06.400356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:06.400356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:06.400474 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:06.400752 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:06.400757 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:06.400916 (+ 159us) client_negotiation.cc:769] Sending connection context
0926 06:55:06.401585 (+ 669us) client_negotiation.cc:240] Negotiation successful
0926 06:55:06.401738 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":84,"threads_started":1}
I20260926 06:55:06.402715 13350 heartbeater.cc:347] Connected to a master server at 127.0.191.254:45911
I20260926 06:55:06.402802 13350 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:06.402982 13350 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
I20260926 06:55:06.403352 13001 ts_manager.cc:194] Registered new tserver with Master: 91e1bb2caa5040b59d7d092cf7a0f600 (127.0.191.194:43135)
I20260926 06:55:06.403808 13001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:56539
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:55:06.428136 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.191.212:46143/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:45911
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:06.553083 13355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:06.553488 13355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:06.553591 13355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:06.555367 13355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:06.555480 13355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:06.555567 13355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:55:06.557873 13355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.191.212:46143/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:06.558368 13355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:06.558712 13355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:06.565479 13362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:06.565893 13364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:06.566725 13355 server_base.cc:1034] running on GCE node
W20260926 06:55:06.565575 13361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:06.573689 13355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:06.574012 13355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:06.581461 13355 hybrid_clock.cc:648] HybridClock initialized: now 1790405706581420 us; error 52 us; skew 500 ppm
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:06.584874 13355 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:55:06.585470 13355 webserver.cc:460] Webserver started at http://127.0.191.195:44919/ using document root <none> and password file <none>
I20260926 06:55:06.585723 13355 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:06.585783 13355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:06.585911 13355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:55:06.607040 13355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "3fcb3582fd4b4803805c8e87b1f6833c"
format_stamp: "Formatted at 2026-09-26 06:55:06 on dist-test-slave-ws74"
server_key: "af834419b091f8b8a5f282ba1be7933f"
server_key_iv: "342ef713cfd13e0734b7d2fe78f70096"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:06.607612 13355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "3fcb3582fd4b4803805c8e87b1f6833c"
format_stamp: "Formatted at 2026-09-26 06:55:06 on dist-test-slave-ws74"
server_key: "af834419b091f8b8a5f282ba1be7933f"
server_key_iv: "342ef713cfd13e0734b7d2fe78f70096"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:06.609361 13355 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260926 06:55:06.625506 13371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:06.625679 13355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 06:55:06.625779 13355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "3fcb3582fd4b4803805c8e87b1f6833c"
format_stamp: "Formatted at 2026-09-26 06:55:06 on dist-test-slave-ws74"
server_key: "af834419b091f8b8a5f282ba1be7933f"
server_key_iv: "342ef713cfd13e0734b7d2fe78f70096"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:06.625880 13355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:06.642671 13355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:06.645167 13355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:06.645327 13355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:06.645614 13355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:06.646003 13355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:06.646052 13355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:06.646085 13355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:06.646104 13355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:06.653589 13355 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:43819
I20260926 06:55:06.654052 13355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:55:06.654086 13484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:43819 every 8 connection(s)
I20260926 06:55:06.658479 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 13355
I20260926 06:55:06.658586 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:55:06.658782 767 external_mini_cluster.cc:1428] Setting key 85a96e339abbd2928fd8a89031cdb915
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:06.664978 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:06.655205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:36701 (local address 127.0.191.254:45911)
0926 06:55:06.655251 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:06.655254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:06.655511 (+ 257us) server_negotiation.cc:407] Connection header received
0926 06:55:06.656270 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:06.656272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:06.656287 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:06.656324 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:06.657056 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:06.657742 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:06.659281 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:06.659580 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:06.659665 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:06.662172 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:06.662173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:06.662174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:06.662181 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:06.663555 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:06.663981 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:06.663982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:06.663983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:06.664045 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:06.664310 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:06.664311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:06.664311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:06.664432 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:06.664507 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:06.664799 (+ 292us) server_negotiation.cc:299] Negotiation successful
0926 06:55:06.664879 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:55:06.665197 13487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:06.655204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:45911 (local address 127.0.191.195:36701)
0926 06:55:06.655373 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:06.655421 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:06.656120 (+ 699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:06.656509 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:06.656513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:06.656698 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:06.656953 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:06.656958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:06.657855 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:06.657857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:06.659179 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:06.659182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:06.659701 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:06.659702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:06.659857 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:06.660519 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:06.660530 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:06.662072 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:06.663658 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:06.663665 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:06.663667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:06.663902 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:06.664115 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:06.664116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:06.664116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:06.664235 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:06.664497 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:06.664504 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:06.664655 (+ 151us) client_negotiation.cc:769] Sending connection context
0926 06:55:06.664778 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 06:55:06.664925 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260926 06:55:06.665635 13485 heartbeater.cc:347] Connected to a master server at 127.0.191.254:45911
I20260926 06:55:06.665736 13485 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:06.665921 13485 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
I20260926 06:55:06.666267 13001 ts_manager.cc:194] Registered new tserver with Master: 3fcb3582fd4b4803805c8e87b1f6833c (127.0.191.195:43819)
I20260926 06:55:06.666641 13001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:36701
I20260926 06:55:06.671801 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:55:06.672502 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 13082
I20260926 06:55:06.677852 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 13219
I20260926 06:55:06.681952 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 13355
I20260926 06:55:06.687071 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 12969
I20260926 06:55:06.692471 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:45911
--webserver_interface=127.0.191.254
--webserver_port=34091
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:45911
--ranger_config_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.191.212:46143/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.191.212:46143/kms with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:06.773761 13488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:06.773978 13488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:06.774011 13488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:06.775199 13488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:55:06.775265 13488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:06.775290 13488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:06.775310 13488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:55:06.775341 13488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:55:06.776880 13488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.191.212:46143/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:45911
--ranger_config_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:45911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=34091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:06.777139 13488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:06.777396 13488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:06.782006 13494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:06.782048 13493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:06.782006 13496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:06.782104 13488 server_base.cc:1034] running on GCE node
I20260926 06:55:06.782461 13488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:06.782735 13488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:06.783911 13488 hybrid_clock.cc:648] HybridClock initialized: now 1790405706783900 us; error 29 us; skew 500 ppm
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:06.786362 13488 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:55:06.786782 13488 webserver.cc:460] Webserver started at http://127.0.191.254:34091/ using document root <none> and password file <none>
I20260926 06:55:06.786962 13488 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:06.787006 13488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:55:06.827282 13488 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:06.828191 13504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:06.828397 13488 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:06.828501 13488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "8ec8fd56f99744abb2d9de674930e6d2"
format_stamp: "Formatted at 2026-09-26 06:55:05 on dist-test-slave-ws74"
server_key: "bb34bdc01c88dae8edd1e3e5852cb507"
server_key_iv: "4eb5a2c8ee3917bc8911dba50cf568a1"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:06.828985 13488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:06.848558 13488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:06.850328 13488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:06.850450 13488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:06.854972 13488 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:45911
I20260926 06:55:06.855021 13556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:45911 every 8 connection(s)
I20260926 06:55:06.855418 13488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:55:06.856566 13557 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:55:06.857221 13557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2: Bootstrap starting.
I20260926 06:55:06.857640 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 13488
I20260926 06:55:06.857926 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.191.212:46143/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:33311
--local_ip_for_outbound_sockets=127.0.191.193
--tserver_master_addrs=127.0.191.254:45911
--webserver_port=35031
--webserver_interface=127.0.191.193
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.191.212:46143/kms with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:55:06.865110 13557 log.cc:824] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:06.866824 13557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:55:06.867112 13557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2: Bootstrap complete.
I20260926 06:55:06.869464 13557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec8fd56f99744abb2d9de674930e6d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45911 } }
I20260926 06:55:06.869738 13557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ec8fd56f99744abb2d9de674930e6d2, State: Initialized, Role: FOLLOWER
I20260926 06:55:06.869941 13557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec8fd56f99744abb2d9de674930e6d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45911 } }
I20260926 06:55:06.870035 13557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:55:06.870086 13557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:55:06.870133 13557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 1 FOLLOWER]: Advancing to term 2
I20260926 06:55:06.871093 13557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec8fd56f99744abb2d9de674930e6d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45911 } }
I20260926 06:55:06.871213 13557 leader_election.cc:302] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ec8fd56f99744abb2d9de674930e6d2; no voters:
I20260926 06:55:06.871417 13557 leader_election.cc:288] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 06:55:06.871454 13561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 06:55:06.871555 13561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [term 2 LEADER]: Becoming Leader. State: Replica: 8ec8fd56f99744abb2d9de674930e6d2, State: Running, Role: LEADER
I20260926 06:55:06.871639 13561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec8fd56f99744abb2d9de674930e6d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45911 } }
I20260926 06:55:06.871678 13557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:55:06.871886 13561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ec8fd56f99744abb2d9de674930e6d2. Latest consensus state: current_term: 2 leader_uuid: "8ec8fd56f99744abb2d9de674930e6d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec8fd56f99744abb2d9de674930e6d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45911 } } }
I20260926 06:55:06.871955 13561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:06.871891 13562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8ec8fd56f99744abb2d9de674930e6d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec8fd56f99744abb2d9de674930e6d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45911 } } }
I20260926 06:55:06.872129 13562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2 [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:06.872339 13565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:55:06.872886 13565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:55:06.873111 13565 catalog_manager.cc:1259] Loaded cluster ID: 805d1afc9a2940b18d9315ba9d2516f5
I20260926 06:55:06.873198 13565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:55:06.873476 13557 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 06:55:06.876652 13565 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:55:06.877233 13565 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2: Loaded TSK: 0
I20260926 06:55:06.877576 13565 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 06:55:06.970295 13559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:06.970602 13559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:06.970643 13559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:06.972452 13559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:06.972517 13559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:06.972561 13559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:55:06.974852 13559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.191.212:46143/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:33311
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=35031
--tserver_master_addrs=127.0.191.254:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:06.975095 13559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:06.975366 13559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:06.982286 13597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:06.982601 13594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:06.983551 13595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:06.984223 13559 server_base.cc:1034] running on GCE node
I20260926 06:55:06.984500 13559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:06.984790 13559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:06.985985 13559 hybrid_clock.cc:648] HybridClock initialized: now 1790405706985954 us; error 52 us; skew 500 ppm
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:06.989554 13559 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:55:06.990132 13559 webserver.cc:460] Webserver started at http://127.0.191.193:35031/ using document root <none> and password file <none>
I20260926 06:55:06.990437 13559 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:06.990531 13559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 06:55:06 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:55:07.013841 13559 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260926 06:55:07.015825 13605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:07.016019 13559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260926 06:55:07.016113 13559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d"
format_stamp: "Formatted at 2026-09-26 06:55:06 on dist-test-slave-ws74"
server_key: "f415dd420839480c4eb860c7a2c22bcd"
server_key_iv: "5cedc64b1c1d515f46d554923b47d04d"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:07.016626 13559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:07.036024 13559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:07.038601 13559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:07.038781 13559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:07.039074 13559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:07.039469 13559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:07.039528 13559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:07.039569 13559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:07.039594 13559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:07.047614 13559 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:33311
I20260926 06:55:07.048095 13559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:55:07.053584 13718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:33311 every 8 connection(s)
I20260926 06:55:07.056627 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 13559
I20260926 06:55:07.056938 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.191.212:46143/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:43135
--local_ip_for_outbound_sockets=127.0.191.194
--tserver_master_addrs=127.0.191.254:45911
--webserver_port=46491
--webserver_interface=127.0.191.194
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.191.212:46143/kms with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 06:55:07 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:07.074940 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:07.050677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:45911 (local address 127.0.191.193:54253)
0926 06:55:07.050987 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:07.051034 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:07.051700 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:07.056675 (+ 4975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:07.056681 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:07.056982 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:07.057288 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:07.057295 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.059230 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.059231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:07.060585 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:07.060588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.061544 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.061545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:07.061698 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:07.062356 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:07.062367 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:07.064118 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:07.072821 (+ 8703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:07.072831 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:07.072833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:07.073180 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:07.073516 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:07.073516 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:07.073517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:07.073691 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:07.074193 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:07.074203 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:07.074362 (+ 159us) client_negotiation.cc:769] Sending connection context
0926 06:55:07.074467 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 06:55:07.074654 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":86,"threads_started":1}
I20260926 06:55:07.075845 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:07.053473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:54253 (local address 127.0.191.254:45911)
0926 06:55:07.056142 (+ 2669us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:07.056153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:07.056212 (+ 59us) server_negotiation.cc:407] Connection header received
0926 06:55:07.056376 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:07.056383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:07.056444 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:07.057463 (+ 1019us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:07.057553 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.059080 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.060709 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.061416 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.062030 (+ 614us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:07.064284 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:07.064289 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:07.064289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:07.064303 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:07.072694 (+ 8391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:07.073280 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:07.073283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:07.073284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:07.073432 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:07.073773 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:07.073773 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:07.073774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:07.074095 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:07.075296 (+ 1201us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:07.075397 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 06:55:07.075563 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2608,"thread_start_us":78,"threads_started":1}
I20260926 06:55:07.078943 13719 heartbeater.cc:347] Connected to a master server at 127.0.191.254:45911
I20260926 06:55:07.079049 13719 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:07.079296 13719 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
W20260926 06:55:07.080156 13719 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:45911 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 06:55:07.080240 13719 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:07.080308 13719 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
I20260926 06:55:07.080585 13719 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:07.080655 13719 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
W20260926 06:55:07.080900 13719 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 06:55:07.178149 13723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:07.178421 13723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:07.178455 13723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:07.180198 13723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:07.180250 13723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:07.180289 13723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:55:07.182425 13723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.191.212:46143/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:43135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=46491
--tserver_master_addrs=127.0.191.254:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:07.182657 13723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:07.182919 13723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:07.188055 13728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:07.188061 13731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:07.188359 13729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:07.188699 13723 server_base.cc:1034] running on GCE node
I20260926 06:55:07.188931 13723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:07.189239 13723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:07.190441 13723 hybrid_clock.cc:648] HybridClock initialized: now 1790405707190429 us; error 44 us; skew 500 ppm
Sep 26 06:55:07 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405707, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:07.193704 13723 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:55:07.194176 13723 webserver.cc:460] Webserver started at http://127.0.191.194:46491/ using document root <none> and password file <none>
I20260926 06:55:07.194456 13723 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:07.194511 13723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 06:55:07 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405707, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:55:07.219015 13723 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260926 06:55:07.220434 13739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:07.220635 13723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:55:07.220722 13723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "91e1bb2caa5040b59d7d092cf7a0f600"
format_stamp: "Formatted at 2026-09-26 06:55:06 on dist-test-slave-ws74"
server_key: "2c3b43a291fc55b03c15841a9ab9fa87"
server_key_iv: "ae3478efac000faeb52da4dc9b5a503d"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:07.221058 13723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:07.247601 13723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:07.249815 13723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:07.249979 13723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:07.250247 13723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:07.250617 13723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:07.250672 13723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:07.250712 13723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:07.250735 13723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:07.257795 13723 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:43135
I20260926 06:55:07.257903 13852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:43135 every 8 connection(s)
I20260926 06:55:07.258258 13723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:55:07.263634 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 13723
I20260926 06:55:07.263967 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.191.212:46143/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:43819
--local_ip_for_outbound_sockets=127.0.191.195
--tserver_master_addrs=127.0.191.254:45911
--webserver_port=44919
--webserver_interface=127.0.191.195
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.191.212:46143/kms with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 06:55:07 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405707, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:07.269490 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:07.259085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:47273 (local address 127.0.191.254:45911)
0926 06:55:07.259142 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:07.259144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:07.259398 (+ 254us) server_negotiation.cc:407] Connection header received
0926 06:55:07.260138 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:07.260140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:07.260152 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:07.260222 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:07.261019 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.261582 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.263275 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.263617 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.263689 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:07.266460 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:07.266461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:07.266461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:07.266469 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:07.267946 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:07.268424 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:07.268425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:07.268426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:07.268486 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:07.268784 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:07.268785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:07.268786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:07.268900 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:07.268976 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:07.269284 (+ 308us) server_negotiation.cc:299] Negotiation successful
0926 06:55:07.269404 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 06:55:07.269665 13855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:07.259010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:45911 (local address 127.0.191.194:47273)
0926 06:55:07.259218 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:07.259270 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:07.260003 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:07.260309 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:07.260314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:07.260579 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:07.260909 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:07.260916 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.261690 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.261692 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:07.263167 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:07.263170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.263841 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.263843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:07.263979 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:07.264818 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:07.264828 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:07.266348 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:07.268047 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:07.268054 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:07.268056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:07.268334 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:07.268565 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:07.268566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:07.268566 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:07.268700 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:07.268967 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:07.268973 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:07.269112 (+ 139us) client_negotiation.cc:769] Sending connection context
0926 06:55:07.269244 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 06:55:07.269400 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
I20260926 06:55:07.270346 13853 heartbeater.cc:347] Connected to a master server at 127.0.191.254:45911
I20260926 06:55:07.270467 13853 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:07.270642 13853 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
W20260926 06:55:07.271323 13853 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:45911 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 06:55:07.271399 13853 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:07.271478 13853 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
I20260926 06:55:07.271770 13853 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:07.271845 13853 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
W20260926 06:55:07.272113 13853 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 06:55:07.384305 13856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:07.384648 13856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:07.384713 13856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:07.386656 13856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:07.386735 13856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:07.386780 13856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:55:07.389220 13856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.191.212:46143/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:43819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=44919
--tserver_master_addrs=127.0.191.254:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:07.389510 13856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:07.389796 13856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:07.396121 13863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:07.396154 13865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:07.396543 13862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:07.397090 13856 server_base.cc:1034] running on GCE node
I20260926 06:55:07.397368 13856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:07.397709 13856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:07.398919 13856 hybrid_clock.cc:648] HybridClock initialized: now 1790405707398888 us; error 44 us; skew 500 ppm
Sep 26 06:55:07 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405707, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:07.403313 13856 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:55:07.403883 13856 webserver.cc:460] Webserver started at http://127.0.191.195:44919/ using document root <none> and password file <none>
I20260926 06:55:07.404184 13856 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:07.404275 13856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 06:55:07 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405707, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:55:07.424367 13856 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:07.425290 13873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:07.425475 13856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:07.425570 13856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "3fcb3582fd4b4803805c8e87b1f6833c"
format_stamp: "Formatted at 2026-09-26 06:55:06 on dist-test-slave-ws74"
server_key: "af834419b091f8b8a5f282ba1be7933f"
server_key_iv: "342ef713cfd13e0734b7d2fe78f70096"
server_key_version: "kuduclusterkey@0"
I20260926 06:55:07.426038 13856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:07.456873 13856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:07.459080 13856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:07.459240 13856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:07.459512 13856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:07.459847 13856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:07.459880 13856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:07.459903 13856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:07.459915 13856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:07.466674 13856 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:43819
I20260926 06:55:07.467139 13856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:55:07.468545 13986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:43819 every 8 connection(s)
I20260926 06:55:07.471156 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 13856
Sep 26 06:55:07 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405707, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:07.480451 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:07.468259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:59567 (local address 127.0.191.254:45911)
0926 06:55:07.468310 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:07.468312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:07.468566 (+ 254us) server_negotiation.cc:407] Connection header received
0926 06:55:07.469222 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:07.469223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:07.469235 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:07.469293 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:07.470033 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.470805 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.472585 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.472884 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.472930 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:07.477030 (+ 4100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:07.477032 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:07.477032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:07.477039 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:07.478816 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:07.479313 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:07.479315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:07.479315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:07.479376 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:07.479687 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:07.479689 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:07.479689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:07.479852 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:07.479953 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:07.480275 (+ 322us) server_negotiation.cc:299] Negotiation successful
0926 06:55:07.480355 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:55:07.480659 13989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:07.468277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:45911 (local address 127.0.191.195:59567)
0926 06:55:07.468435 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:07.468476 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:07.469098 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:07.469397 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:07.469402 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:07.469645 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:07.469921 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:07.469927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.470945 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.470946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:07.472485 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:07.472488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.473030 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.473031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:07.473166 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:07.473787 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:07.473796 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:07.476912 (+ 3116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:07.478932 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:07.478940 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:07.478942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:07.479249 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:07.479444 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:07.479445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:07.479446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:07.479587 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:07.479951 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:07.479957 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:07.480123 (+ 166us) client_negotiation.cc:769] Sending connection context
0926 06:55:07.480261 (+ 138us) client_negotiation.cc:240] Negotiation successful
0926 06:55:07.480414 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260926 06:55:07.481168 13987 heartbeater.cc:347] Connected to a master server at 127.0.191.254:45911
I20260926 06:55:07.481273 13987 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:07.481483 13987 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
W20260926 06:55:07.482177 13987 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:45911 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 06:55:07.482267 13987 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:07.482340 13987 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
I20260926 06:55:07.482719 13987 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:07.482810 13987 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
W20260926 06:55:07.483141 13987 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 06:55:07.486593 13990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:07.471585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53632 (local address 127.0.191.254:45911)
0926 06:55:07.471718 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:07.471720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:07.476268 (+ 4548us) server_negotiation.cc:407] Connection header received
0926 06:55:07.476294 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:07.476295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:07.476309 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:07.476383 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:07.479325 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.480004 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.481656 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:07.481960 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:07.482035 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:07.483556 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:07.483557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:07.483558 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:07.483566 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:07.485351 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:07.485713 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:07.485714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:07.485715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:07.485782 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:07.485979 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:07.485980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:07.485980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:07.486126 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:07.486324 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:07.486384 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 06:55:07.486477 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
Sep 26 06:55:08 dist-test-slave-ws74 krb5kdc[12525](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790405708, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:08.081084 13719 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:08.081247 13719 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
I20260926 06:55:08.081636 13521 ts_manager.cc:194] Registered new tserver with Master: 4cc3f6ecd1b140e3a354fec8fcb9c80d (127.0.191.193:33311)
I20260926 06:55:08.082327 13521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:54253
I20260926 06:55:08.272305 13853 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:08.272456 13853 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
I20260926 06:55:08.275287 13520 ts_manager.cc:194] Registered new tserver with Master: 91e1bb2caa5040b59d7d092cf7a0f600 (127.0.191.194:43135)
I20260926 06:55:08.275825 13520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:47273
Sep 26 06:55:08 dist-test-slave-ws74 krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405708, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for HTTP/127.0.191.212@KRBTEST.COM
I20260926 06:55:08.483354 13987 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:08.483502 13987 heartbeater.cc:511] Master 127.0.191.254:45911 requested a full tablet report, sending...
I20260926 06:55:08.483810 13520 ts_manager.cc:194] Registered new tserver with Master: 3fcb3582fd4b4803805c8e87b1f6833c (127.0.191.195:43819)
I20260926 06:55:08.484346 13520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:59567
I20260926 06:55:08.493798 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:55:08.503461 14009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.494716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53644 (local address 127.0.191.254:45911)
0926 06:55:08.494921 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:08.494923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:08.494930 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:55:08.494950 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:08.494950 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:08.494961 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:08.495019 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:08.495737 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.496520 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.497580 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.497903 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.497980 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:08.499423 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:08.499425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:08.499425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:08.499433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:08.501449 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:08.501858 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:08.501860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:08.501860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:08.501935 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:08.502215 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:08.502215 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:08.502216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:08.502365 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:08.502475 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:08.502625 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 06:55:08.503226 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":55,"threads_started":1}
I20260926 06:55:08.506558 13520 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:55:08.508904 13520 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:55:08.516779 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.511661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:43819 (local address 127.0.0.1:52994)
0926 06:55:08.512143 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:08.512183 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:08.512252 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:08.512740 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:08.512742 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:08.512744 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:08.512799 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:08.512800 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.514162 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.514163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:08.515322 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:08.515324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.516244 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.516245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:08.516319 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:08.516332 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 06:55:08.516387 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 06:55:08.516699 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":20,"spinlock_wait_cycles":768,"thread_start_us":90,"threads_started":1}
I20260926 06:55:08.517131 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.511772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52994 (local address 127.0.191.195:43819)
0926 06:55:08.512393 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:08.512398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:08.512409 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:55:08.512444 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:08.512447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:08.512462 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:08.512533 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:08.512898 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.514072 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.515432 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.516149 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.516896 (+ 747us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:08.516987 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:08.517017 (+ 30us) server_negotiation.cc:299] Negotiation successful
0926 06:55:08.517057 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":63,"threads_started":1}
I20260926 06:55:08.517880 13921 tablet_service.cc:1467] Processing CreateTablet for tablet c70e0b6beda34eaab4348787701af809 (DEFAULT_TABLE table=test-table [id=e080d668939c478a8de1d4684a8150f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:08.518021 14017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.511893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:43135 (local address 127.0.0.1:44176)
0926 06:55:08.513004 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:08.513014 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:08.513058 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:08.513302 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:08.513302 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:08.513304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:08.513337 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:08.513338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.514526 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.514527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:08.516396 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:08.516397 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.517804 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.517805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:08.517875 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:08.517879 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:55:08.517930 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 06:55:08.517963 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"mutex_wait_us":2,"thread_start_us":63,"threads_started":1}
I20260926 06:55:08.518258 13921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c70e0b6beda34eaab4348787701af809. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:08.518307 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.512001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44176 (local address 127.0.191.194:43135)
0926 06:55:08.512947 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:08.512953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:08.513108 (+ 155us) server_negotiation.cc:407] Connection header received
0926 06:55:08.513146 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:08.513149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:08.513165 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:08.513232 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:08.513432 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.514444 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.516497 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.517650 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.517716 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:08.517770 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:08.518178 (+ 408us) server_negotiation.cc:299] Negotiation successful
0926 06:55:08.518223 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":909,"thread_start_us":46,"threads_started":1}
I20260926 06:55:08.518846 13780 tablet_service.cc:1467] Processing CreateTablet for tablet c70e0b6beda34eaab4348787701af809 (DEFAULT_TABLE table=test-table [id=e080d668939c478a8de1d4684a8150f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:08.519191 13780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c70e0b6beda34eaab4348787701af809. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:08.523231 14021 tablet_bootstrap.cc:492] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600: Bootstrap starting.
I20260926 06:55:08.523945 14021 tablet_bootstrap.cc:654] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:08.524789 14021 log.cc:824] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:08.525280 14020 tablet_bootstrap.cc:492] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c: Bootstrap starting.
I20260926 06:55:08.525942 14020 tablet_bootstrap.cc:654] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:08.526300 14020 log.cc:824] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:08.527058 14021 tablet_bootstrap.cc:492] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600: No bootstrap required, opened a new log
I20260926 06:55:08.527170 14021 ts_tablet_manager.cc:1397] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 06:55:08.527559 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.511659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:33311 (local address 127.0.0.1:37110)
0926 06:55:08.512629 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:08.512656 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:08.512691 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:08.523009 (+ 10318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:08.523010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:08.523012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:08.523050 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:08.523052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.525222 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.525224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:08.526356 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:08.526358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.527325 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.527327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:08.527386 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:08.527390 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:55:08.527420 (+ 30us) client_negotiation.cc:240] Negotiation successful
0926 06:55:08.527468 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":93,"threads_started":1}
I20260926 06:55:08.527895 14019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.512083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37110 (local address 127.0.191.193:33311)
0926 06:55:08.522849 (+ 10766us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:08.522853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:08.522869 (+ 16us) server_negotiation.cc:407] Connection header received
0926 06:55:08.522896 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:08.522897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:08.522909 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:08.523012 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:08.523125 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.525109 (+ 1984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.526460 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.527218 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.527669 (+ 451us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:08.527754 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:08.527778 (+ 24us) server_negotiation.cc:299] Negotiation successful
0926 06:55:08.527817 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10737,"thread_start_us":50,"threads_started":1}
I20260926 06:55:08.528609 13653 tablet_service.cc:1467] Processing CreateTablet for tablet c70e0b6beda34eaab4348787701af809 (DEFAULT_TABLE table=test-table [id=e080d668939c478a8de1d4684a8150f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:08.528923 13653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c70e0b6beda34eaab4348787701af809. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:08.530156 14021 raft_consensus.cc:348] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33311 } } peers { permanent_uuid: "3fcb3582fd4b4803805c8e87b1f6833c" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43819 } } peers { permanent_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43135 } }
I20260926 06:55:08.530330 14021 raft_consensus.cc:374] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:08.530364 14021 raft_consensus.cc:729] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e1bb2caa5040b59d7d092cf7a0f600, State: Initialized, Role: FOLLOWER
I20260926 06:55:08.530488 14021 consensus_queue.cc:260] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33311 } } peers { permanent_uuid: "3fcb3582fd4b4803805c8e87b1f6833c" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43819 } } peers { permanent_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43135 } }
I20260926 06:55:08.530735 14021 ts_tablet_manager.cc:1428] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:55:08.530843 13853 heartbeater.cc:503] Master 127.0.191.254:45911 was elected leader, sending a full tablet report...
I20260926 06:55:08.532171 14020 tablet_bootstrap.cc:492] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c: No bootstrap required, opened a new log
I20260926 06:55:08.532253 14020 ts_tablet_manager.cc:1397] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260926 06:55:08.534547 14020 raft_consensus.cc:348] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33311 } } peers { permanent_uuid: "3fcb3582fd4b4803805c8e87b1f6833c" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43819 } } peers { permanent_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43135 } }
I20260926 06:55:08.534737 14020 raft_consensus.cc:374] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:08.534778 14020 raft_consensus.cc:729] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcb3582fd4b4803805c8e87b1f6833c, State: Initialized, Role: FOLLOWER
I20260926 06:55:08.534926 14020 consensus_queue.cc:260] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33311 } } peers { permanent_uuid: "3fcb3582fd4b4803805c8e87b1f6833c" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43819 } } peers { permanent_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43135 } }
I20260926 06:55:08.535213 13987 heartbeater.cc:503] Master 127.0.191.254:45911 was elected leader, sending a full tablet report...
I20260926 06:55:08.535203 14020 ts_tablet_manager.cc:1428] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:55:08.536088 14025 tablet_bootstrap.cc:492] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d: Bootstrap starting.
I20260926 06:55:08.536679 14025 tablet_bootstrap.cc:654] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:08.536969 14025 log.cc:824] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:08.537711 14025 tablet_bootstrap.cc:492] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d: No bootstrap required, opened a new log
I20260926 06:55:08.537825 14025 ts_tablet_manager.cc:1397] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:55:08.539559 14025 raft_consensus.cc:348] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33311 } } peers { permanent_uuid: "3fcb3582fd4b4803805c8e87b1f6833c" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43819 } } peers { permanent_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43135 } }
I20260926 06:55:08.539702 14025 raft_consensus.cc:374] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:08.539772 14025 raft_consensus.cc:729] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc3f6ecd1b140e3a354fec8fcb9c80d, State: Initialized, Role: FOLLOWER
I20260926 06:55:08.539884 14025 consensus_queue.cc:260] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33311 } } peers { permanent_uuid: "3fcb3582fd4b4803805c8e87b1f6833c" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43819 } } peers { permanent_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43135 } }
I20260926 06:55:08.540222 13719 heartbeater.cc:503] Master 127.0.191.254:45911 was elected leader, sending a full tablet report...
I20260926 06:55:08.540213 14025 ts_tablet_manager.cc:1428] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260926 06:55:08.551017 13720 tablet.cc:2369] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:55:08.616353 14024 raft_consensus.cc:482] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:55:08.616519 14024 raft_consensus.cc:504] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33311 } } peers { permanent_uuid: "3fcb3582fd4b4803805c8e87b1f6833c" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43819 } } peers { permanent_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43135 } }
I20260926 06:55:08.617220 14024 leader_election.cc:288] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cc3f6ecd1b140e3a354fec8fcb9c80d (127.0.191.193:33311), 3fcb3582fd4b4803805c8e87b1f6833c (127.0.191.195:43819)
I20260926 06:55:08.621031 14019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.617465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:59223 (local address 127.0.191.193:33311)
0926 06:55:08.617575 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:08.617577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:08.617694 (+ 117us) server_negotiation.cc:407] Connection header received
0926 06:55:08.617777 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:08.617778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:08.617787 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:08.617814 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:08.618107 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.618877 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.620151 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.620665 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.620709 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:08.620716 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:08.620925 (+ 209us) server_negotiation.cc:299] Negotiation successful
0926 06:55:08.620964 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 06:55:08.621129 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.617414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:59955 (local address 127.0.191.195:43819)
0926 06:55:08.617515 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:08.617517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:08.617527 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:55:08.617555 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:08.617556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:08.617565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:08.617627 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:08.617883 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.618625 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.620048 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.620637 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.620690 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:08.620698 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:08.621014 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 06:55:08.621058 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 06:55:08.621353 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.617084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:43819 (local address 127.0.191.194:59955)
0926 06:55:08.617412 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:08.617424 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:08.617479 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:08.617683 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:08.617684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:08.617686 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:08.617791 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:08.617794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.618726 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.618738 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:08.619941 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:08.619944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.620806 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.620807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:08.620881 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:08.620884 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:55:08.621235 (+ 351us) client_negotiation.cc:240] Negotiation successful
0926 06:55:08.621276 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":11,"thread_start_us":100,"threads_started":1}
I20260926 06:55:08.621656 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.617093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:33311 (local address 127.0.191.194:59223)
0926 06:55:08.617649 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:08.617657 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:08.617713 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:08.617873 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:08.617874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:08.617875 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:08.617999 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:08.618002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.618982 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.618983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:08.620055 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:08.620058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.620771 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.620772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:08.620844 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:08.620847 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:55:08.621413 (+ 566us) client_negotiation.cc:240] Negotiation successful
0926 06:55:08.621566 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":113,"threads_started":1}
I20260926 06:55:08.621681 13941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c70e0b6beda34eaab4348787701af809" candidate_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcb3582fd4b4803805c8e87b1f6833c" is_pre_election: true
I20260926 06:55:08.621865 13941 raft_consensus.cc:2454] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e1bb2caa5040b59d7d092cf7a0f600 in term 0.
I20260926 06:55:08.622162 13742 leader_election.cc:302] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcb3582fd4b4803805c8e87b1f6833c, 91e1bb2caa5040b59d7d092cf7a0f600; no voters:
I20260926 06:55:08.622143 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c70e0b6beda34eaab4348787701af809" candidate_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d" is_pre_election: true
I20260926 06:55:08.622303 14024 raft_consensus.cc:2781] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:55:08.622304 13671 raft_consensus.cc:2454] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e1bb2caa5040b59d7d092cf7a0f600 in term 0.
I20260926 06:55:08.622351 14024 raft_consensus.cc:482] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:55:08.622376 14024 raft_consensus.cc:3037] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:08.623317 14024 raft_consensus.cc:504] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33311 } } peers { permanent_uuid: "3fcb3582fd4b4803805c8e87b1f6833c" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43819 } } peers { permanent_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43135 } }
I20260926 06:55:08.623471 14024 leader_election.cc:288] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [CANDIDATE]: Term 1 election: Requested vote from peers 4cc3f6ecd1b140e3a354fec8fcb9c80d (127.0.191.193:33311), 3fcb3582fd4b4803805c8e87b1f6833c (127.0.191.195:43819)
I20260926 06:55:08.623657 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c70e0b6beda34eaab4348787701af809" candidate_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d"
I20260926 06:55:08.623703 13941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c70e0b6beda34eaab4348787701af809" candidate_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcb3582fd4b4803805c8e87b1f6833c"
I20260926 06:55:08.623764 13671 raft_consensus.cc:3037] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:08.623792 13941 raft_consensus.cc:3037] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:08.624509 13671 raft_consensus.cc:2454] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e1bb2caa5040b59d7d092cf7a0f600 in term 1.
I20260926 06:55:08.624679 13740 leader_election.cc:302] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cc3f6ecd1b140e3a354fec8fcb9c80d, 91e1bb2caa5040b59d7d092cf7a0f600; no voters:
I20260926 06:55:08.624691 13941 raft_consensus.cc:2454] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e1bb2caa5040b59d7d092cf7a0f600 in term 1.
I20260926 06:55:08.624888 14024 raft_consensus.cc:2781] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:55:08.625048 14024 raft_consensus.cc:686] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [term 1 LEADER]: Becoming Leader. State: Replica: 91e1bb2caa5040b59d7d092cf7a0f600, State: Running, Role: LEADER
I20260926 06:55:08.625144 14024 consensus_queue.cc:237] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33311 } } peers { permanent_uuid: "3fcb3582fd4b4803805c8e87b1f6833c" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43819 } } peers { permanent_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43135 } }
I20260926 06:55:08.625921 13520 catalog_manager.cc:5495] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91e1bb2caa5040b59d7d092cf7a0f600 (127.0.191.194). New cstate: current_term: 1 leader_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcb3582fd4b4803805c8e87b1f6833c" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e1bb2caa5040b59d7d092cf7a0f600" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 43135 } health_report { overall_health: HEALTHY } } }
I20260926 06:55:08.637009 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.633443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44184 (local address 127.0.191.194:43135)
0926 06:55:08.633488 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:08.633490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:08.633500 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:55:08.633536 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:08.633537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:08.633557 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:08.633637 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:55:08.633838 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.634522 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.635824 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:08.636134 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:08.636199 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:08.636511 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:55:08.636641 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:55:08.636718 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:08.636904 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 06:55:08.636925 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:55:08.639225 13941 raft_consensus.cc:1261] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c [term 1 FOLLOWER]: Refusing update from remote peer 91e1bb2caa5040b59d7d092cf7a0f600: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:55:08.639757 13671 raft_consensus.cc:1261] T c70e0b6beda34eaab4348787701af809 P 4cc3f6ecd1b140e3a354fec8fcb9c80d [term 1 FOLLOWER]: Refusing update from remote peer 91e1bb2caa5040b59d7d092cf7a0f600: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:55:08.640012 14024 consensus_queue.cc:1035] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcb3582fd4b4803805c8e87b1f6833c" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:55:08.640275 14031 consensus_queue.cc:1035] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cc3f6ecd1b140e3a354fec8fcb9c80d" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:55:08.643563 14034 mvcc.cc:205] Tried to move back new op lower bound from 7333501782583382016 to 7333501782529089536. Current Snapshot: MvccSnapshot[applied={T|T < 7333501782583382016}]
I20260926 06:55:08.644073 14035 mvcc.cc:205] Tried to move back new op lower bound from 7333501782583382016 to 7333501782529089536. Current Snapshot: MvccSnapshot[applied={T|T < 7333501782583382016}]
I20260926 06:55:08.651944 13520 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:55:08.652051 13520 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:55:08.652887 13520 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ec8fd56f99744abb2d9de674930e6d2: Sending DeleteTablet for 3 replicas of tablet c70e0b6beda34eaab4348787701af809
I20260926 06:55:08.653451 13653 tablet_service.cc:1514] Processing DeleteTablet for tablet c70e0b6beda34eaab4348787701af809 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:08 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:37110
I20260926 06:55:08.653688 13921 tablet_service.cc:1514] Processing DeleteTablet for tablet c70e0b6beda34eaab4348787701af809 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:08 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:52994
I20260926 06:55:08.654142 13780 tablet_service.cc:1514] Processing DeleteTablet for tablet c70e0b6beda34eaab4348787701af809 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:08 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:44176
I20260926 06:55:08.654641 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 13559
I20260926 06:55:08.657521 14043 tablet_replica.cc:330] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c: stopping tablet replica
I20260926 06:55:08.657681 14043 raft_consensus.cc:2229] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:55:08.657855 14043 raft_consensus.cc:2258] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:55:08.658125 14044 tablet_replica.cc:330] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600: stopping tablet replica
I20260926 06:55:08.658269 14044 raft_consensus.cc:2229] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:55:08.658398 14044 raft_consensus.cc:2258] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:55:08.658448 14043 ts_tablet_manager.cc:1905] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:55:08.658916 14044 ts_tablet_manager.cc:1905] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:55:08.660635 14044 ts_tablet_manager.cc:1918] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:55:08.660764 14044 log.cc:1196] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/c70e0b6beda34eaab4348787701af809
I20260926 06:55:08.661070 14044 ts_tablet_manager.cc:1939] T c70e0b6beda34eaab4348787701af809 P 91e1bb2caa5040b59d7d092cf7a0f600: Deleting consensus metadata
I20260926 06:55:08.661549 13505 catalog_manager.cc:4841] TS 91e1bb2caa5040b59d7d092cf7a0f600 (127.0.191.194:43135): tablet c70e0b6beda34eaab4348787701af809 (table test-table [id=e080d668939c478a8de1d4684a8150f2]) successfully deleted
I20260926 06:55:08.661778 14043 ts_tablet_manager.cc:1918] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:55:08.661885 14043 log.cc:1196] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/c70e0b6beda34eaab4348787701af809
I20260926 06:55:08.662170 14043 ts_tablet_manager.cc:1939] T c70e0b6beda34eaab4348787701af809 P 3fcb3582fd4b4803805c8e87b1f6833c: Deleting consensus metadata
I20260926 06:55:08.662590 13507 catalog_manager.cc:4841] TS 3fcb3582fd4b4803805c8e87b1f6833c (127.0.191.195:43819): tablet c70e0b6beda34eaab4348787701af809 (table test-table [id=e080d668939c478a8de1d4684a8150f2]) successfully deleted
W20260926 06:55:08.669013 13740 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:33311 (error 108)
W20260926 06:55:08.669409 13505 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:33311 (error 108)
I20260926 06:55:08.669919 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 13723
I20260926 06:55:08.669936 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.669792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:33311 (local address 127.0.191.194:48931)
0926 06:55:08.669860 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:08.669905 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 06:55:08.673772 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:08.673634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:33311 (local address 127.0.0.1:37122)
0926 06:55:08.673674 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:08.673731 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260926 06:55:08.673914 13505 catalog_manager.cc:4584] TS 4cc3f6ecd1b140e3a354fec8fcb9c80d (127.0.191.193:33311): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c70e0b6beda34eaab4348787701af809: Network error: Client connection negotiation failed: client connection to 127.0.191.193:33311: connect: Connection refused (error 111)
I20260926 06:55:08.676402 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 13856
I20260926 06:55:08.681756 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 13488
I20260926 06:55:08.689015 767 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 06:55:09.023053 767 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 06:55:09.023144 767 mini_ranger.cc:66] Stopping Ranger...
I20260926 06:55:09.367470 767 mini_ranger.cc:68] Stopped Ranger
2026-09-26 06:55:09.367 UTC [12555] LOG: received smart shutdown request
2026-09-26 06:55:09.371 UTC [12555] LOG: background worker "logical replication launcher" (PID 12564) exited with exit code 1
2026-09-26 06:55:09.371 UTC [12559] LOG: shutting down
2026-09-26 06:55:09.531 UTC [12555] LOG: database system is shut down
2026-09-26T06:55:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46249 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:55:09 dist-test-slave-ws74 krb5kdc[14056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:55:09 dist-test-slave-ws74 krb5kdc[14056](info): set up 2 sockets
Sep 26 06:55:09 dist-test-slave-ws74 krb5kdc[14056](info): commencing operation
krb5kdc: starting...
W20260926 06:55:10.677424 767 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:55:10 dist-test-slave-ws74 krb5kdc[14056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:55:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:55:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:55:10.795828 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:45989
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:45989,127.0.191.253:35833,127.0.191.252:37693
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:10.875612 14069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:10.875804 14069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:10.875825 14069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:10.876883 14069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:55:10.876915 14069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:10.876929 14069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:10.876941 14069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:55:10.876955 14069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:55:10.878371 14069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:45989,127.0.191.253:35833,127.0.191.252:37693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:45989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:10.878558 14069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:10.878714 14069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:10.883524 14077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:10.883512 14075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:10.883726 14069 server_base.cc:1034] running on GCE node
W20260926 06:55:10.883507 14074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:10.884087 14069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:10.884399 14069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:10.885561 14069 hybrid_clock.cc:648] HybridClock initialized: now 1790405710885550 us; error 45 us; skew 500 ppm
Sep 26 06:55:10 dist-test-slave-ws74 krb5kdc[14056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405710, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:10.891950 14069 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:55:10.892547 14069 webserver.cc:460] Webserver started at http://127.0.191.254:36739/ using document root <none> and password file <none>
I20260926 06:55:10.892814 14069 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:10.892887 14069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:10.893038 14069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:55:10.894225 14069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "1d845f9deff5461e90e1959a133042de"
format_stamp: "Formatted at 2026-09-26 06:55:10 on dist-test-slave-ws74"
server_key: "4c50faf3e4d0f9dcb515b361f46a369f"
server_key_iv: "7d74d0bda1bdafda6b1c9db936f40fc1"
server_key_version: "encryptionkey@0"
I20260926 06:55:10.894528 14069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "1d845f9deff5461e90e1959a133042de"
format_stamp: "Formatted at 2026-09-26 06:55:10 on dist-test-slave-ws74"
server_key: "4c50faf3e4d0f9dcb515b361f46a369f"
server_key_iv: "7d74d0bda1bdafda6b1c9db936f40fc1"
server_key_version: "encryptionkey@0"
I20260926 06:55:10.895787 14069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:55:10.896528 14084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:10.896747 14069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:55:10.896818 14069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "1d845f9deff5461e90e1959a133042de"
format_stamp: "Formatted at 2026-09-26 06:55:10 on dist-test-slave-ws74"
server_key: "4c50faf3e4d0f9dcb515b361f46a369f"
server_key_iv: "7d74d0bda1bdafda6b1c9db936f40fc1"
server_key_version: "encryptionkey@0"
I20260926 06:55:10.896871 14069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:10.919636 14069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:10.922031 14069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:10.922150 14069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:10.925983 14069 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:45989
I20260926 06:55:10.926036 14136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:45989 every 8 connection(s)
I20260926 06:55:10.926393 14069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:55:10.927233 14137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:10.928810 14137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:55:10.930820 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 14069
I20260926 06:55:10.930955 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:55:10.931193 767 external_mini_cluster.cc:1428] Setting key 667ad0d9cefad3f69f3f994bde401cb5
Sep 26 06:55:10 dist-test-slave-ws74 krb5kdc[14056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405710, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.191.253@KRBTEST.COM; defaulting to no policy
I20260926 06:55:10.940517 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:10.929535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:45989 (local address 127.0.0.1:58818)
0926 06:55:10.929684 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:10.929718 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:10.930152 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:10.930437 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:10.930439 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:10.930674 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:10.930824 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:10.930828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:10.932600 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:10.932601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:10.933772 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:10.933774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:10.934352 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:10.934353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:10.934440 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:10.935055 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:10.935066 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:10.936720 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:10.938757 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:10.938762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:10.938765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:10.938976 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:10.939262 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:10.939263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:10.939263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:10.939357 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:10.939856 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:10.939862 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:10.939929 (+ 67us) client_negotiation.cc:769] Sending connection context
0926 06:55:10.940049 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 06:55:10.940311 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":114,"thread_start_us":64,"threads_started":1}
I20260926 06:55:10.940554 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:10.929681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.0.191.254:45989)
0926 06:55:10.929794 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:10.929798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:10.929834 (+ 36us) server_negotiation.cc:407] Connection header received
0926 06:55:10.930305 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:10.930308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:10.930325 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:10.930407 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:10.930945 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:10.932488 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:10.933870 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:10.934258 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:10.934346 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:10.936829 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:10.936840 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:10.936841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:10.936847 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:10.938610 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:10.939084 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:10.939087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:10.939088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:10.939183 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:10.939429 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:10.939430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:10.939430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:10.939766 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:10.939881 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:10.940084 (+ 203us) server_negotiation.cc:299] Negotiation successful
0926 06:55:10.940242 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":57,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.253@KRBTEST.COM" created.
I20260926 06:55:10.941437 14137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:55:10.941833 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:10.941705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:35833 (local address 127.0.0.1:58060)
0926 06:55:10.941762 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:10.941810 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:35833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260926 06:55:10.941963 14085 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.191.253:35833: connect: Connection refused (error 111)
I20260926 06:55:10.942339 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:10.942247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:35833 (local address 127.0.0.1:58074)
0926 06:55:10.942294 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:10.942316 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:35833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 06:55:10.942446 14137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.253:35833: Network error: Client connection negotiation failed: client connection to 127.0.191.253:35833: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.253.keytab.
Entry for principal kudu/127.0.191.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.253.keytab.
I20260926 06:55:10.956395 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.253:35833
--webserver_interface=127.0.191.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.253.keytab
--principal=kudu/127.0.191.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:45989,127.0.191.253:35833,127.0.191.252:37693
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:55:10.991694 14137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } attempt: 1
I20260926 06:55:10.992223 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:10.992072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:35833 (local address 127.0.0.1:58084)
0926 06:55:10.992140 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:10.992188 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:35833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 06:55:10.992595 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:10.992505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:35833 (local address 127.0.0.1:58096)
0926 06:55:10.992549 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:10.992576 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:35833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 06:55:10.992723 14137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.253:35833: Network error: Client connection negotiation failed: client connection to 127.0.191.253:35833: connect: Connection refused (error 111)
W20260926 06:55:11.036000 14144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:11.036191 14144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:11.036214 14144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:11.037338 14144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:55:11.037403 14144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:11.037457 14144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:11.037487 14144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:55:11.037515 14144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:55:11.038991 14144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:45989,127.0.191.253:35833,127.0.191.252:37693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.253.keytab
--principal=kudu/127.0.191.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.253:35833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.191.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:11.039223 14144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:11.039455 14144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:11.044059 14149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:11.044119 14150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:11.044243 14144 server_base.cc:1034] running on GCE node
W20260926 06:55:11.044355 14152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:11.044593 14144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:11.044839 14144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:11.046001 14144 hybrid_clock.cc:648] HybridClock initialized: now 1790405711045990 us; error 39 us; skew 500 ppm
Sep 26 06:55:11 dist-test-slave-ws74 krb5kdc[14056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405711, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:11.048466 14144 init.cc:377] Logged in from keytab as kudu/127.0.191.253@KRBTEST.COM (short username kudu)
I20260926 06:55:11.048905 14144 webserver.cc:460] Webserver started at http://127.0.191.253:35531/ using document root <none> and password file <none>
I20260926 06:55:11.049141 14144 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:11.049209 14144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:11.049407 14144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:55:11.050463 14144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/data/instance:
uuid: "f37a293f7bc14090b517b1d3bb4a302d"
format_stamp: "Formatted at 2026-09-26 06:55:11 on dist-test-slave-ws74"
server_key: "0136607375425675c20c1a4323ceee11"
server_key_iv: "2f362d41b8fd72f0621246708821ce0a"
server_key_version: "encryptionkey@0"
I20260926 06:55:11.050820 14144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/wal/instance:
uuid: "f37a293f7bc14090b517b1d3bb4a302d"
format_stamp: "Formatted at 2026-09-26 06:55:11 on dist-test-slave-ws74"
server_key: "0136607375425675c20c1a4323ceee11"
server_key_iv: "2f362d41b8fd72f0621246708821ce0a"
server_key_version: "encryptionkey@0"
I20260926 06:55:11.052130 14144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:55:11.052840 14159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:11.053005 14144 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:55:11.053102 14144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/wal
uuid: "f37a293f7bc14090b517b1d3bb4a302d"
format_stamp: "Formatted at 2026-09-26 06:55:11 on dist-test-slave-ws74"
server_key: "0136607375425675c20c1a4323ceee11"
server_key_iv: "2f362d41b8fd72f0621246708821ce0a"
server_key_version: "encryptionkey@0"
I20260926 06:55:11.053181 14144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:11.060907 14137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } attempt: 2
I20260926 06:55:11.061299 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.061173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:35833 (local address 127.0.0.1:58108)
0926 06:55:11.061231 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.061270 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:35833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 06:55:11.061661 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.061579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:35833 (local address 127.0.0.1:58120)
0926 06:55:11.061620 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.061644 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:35833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 06:55:11.061764 14137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.253:35833: Network error: Client connection negotiation failed: client connection to 127.0.191.253:35833: connect: Connection refused (error 111)
I20260926 06:55:11.071868 14144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:11.075289 14144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:11.075438 14144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:11.079372 14144 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.253:35833
I20260926 06:55:11.079402 14211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.253:35833 every 8 connection(s)
I20260926 06:55:11.079757 14144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/data/info.pb
I20260926 06:55:11.080591 14212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:11.080668 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 14144
I20260926 06:55:11.080765 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-1/wal/instance
I20260926 06:55:11.080950 767 external_mini_cluster.cc:1428] Setting key 2b1c4a595f687c5fe826306909e4c43b
I20260926 06:55:11.082468 14212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.191.252@KRBTEST.COM; defaulting to no policy
Sep 26 06:55:11 dist-test-slave-ws74 krb5kdc[14056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405711, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.253@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.252@KRBTEST.COM" created.
I20260926 06:55:11.091877 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.083263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:45989 (local address 127.0.0.1:58828)
0926 06:55:11.083408 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.083448 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:11.083883 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:11.084129 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:11.084132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:11.084440 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:11.084644 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.084648 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.085705 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.085706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.086479 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.086481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.087009 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.087009 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.087110 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.087585 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:11.087592 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:11.088908 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:11.090520 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.090526 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.090528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.090733 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.090972 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.090973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.090973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.091096 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.091416 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:11.091422 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:11.091529 (+ 107us) client_negotiation.cc:769] Sending connection context
0926 06:55:11.091616 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 06:55:11.091724 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
I20260926 06:55:11.091899 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.083300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.0.191.254:45989)
0926 06:55:11.083364 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.083366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.083532 (+ 166us) server_negotiation.cc:407] Connection header received
0926 06:55:11.083977 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.083978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.083993 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.084064 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.084741 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.085618 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.086567 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.086916 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.086991 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.088986 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.088987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.088987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.088994 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.090449 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.090830 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.090830 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.090831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.090887 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.091175 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.091176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.091177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.091341 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.091414 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.091659 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.091748 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 06:55:11.092450 14212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:55:11 dist-test-slave-ws74 krb5kdc[14056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405711, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.253@KRBTEST.COM for kudu/127.0.191.253@KRBTEST.COM
I20260926 06:55:11.099910 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.092730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:35833 (local address 127.0.0.1:58122)
0926 06:55:11.092783 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.092793 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:11.092827 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:11.093064 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:11.093065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:11.093216 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:11.093269 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.093270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.094110 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.094111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.095007 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.095010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.095458 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.095459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.095543 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.095883 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:11.095889 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:11.097111 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:11.098816 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.098817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.098817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.098980 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.099182 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.099183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.099184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.099235 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.099586 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:11.099587 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:11.099624 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 06:55:11.099692 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 06:55:11.099790 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 06:55:11.099990 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.092785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58122 (local address 127.0.191.253:35833)
0926 06:55:11.092936 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.092939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.092953 (+ 14us) server_negotiation.cc:407] Connection header received
0926 06:55:11.092970 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.092972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.092981 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.093040 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.093336 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.094021 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.095083 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.095379 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.095433 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.097195 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.097202 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.097203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.097209 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.098715 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.099058 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.099061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.099061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.099117 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.099306 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.099307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.099308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.099519 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.099608 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.099721 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.099859 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260926 06:55:11.100617 14212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:55:11.100965 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.100844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:37693 (local address 127.0.0.1:35210)
0926 06:55:11.100895 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.100935 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 06:55:11.101105 14163 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
I20260926 06:55:11.101583 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.101478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:37693 (local address 127.0.0.1:35214)
0926 06:55:11.101530 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.101557 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260926 06:55:11.101724 14212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:37693: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.252.keytab.
Entry for principal kudu/127.0.191.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.252.keytab.
I20260926 06:55:11.107697 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.252:37693
--webserver_interface=127.0.191.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.252.keytab
--principal=kudu/127.0.191.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:45989,127.0.191.253:35833,127.0.191.252:37693
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:55:11.150964 14212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } attempt: 1
I20260926 06:55:11.151472 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.151345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:37693 (local address 127.0.0.1:35218)
0926 06:55:11.151403 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.151447 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 06:55:11.151860 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.151780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:37693 (local address 127.0.0.1:35230)
0926 06:55:11.151822 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.151845 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 06:55:11.151978 14212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:37693: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
I20260926 06:55:11.152920 14137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } attempt: 3
Sep 26 06:55:11 dist-test-slave-ws74 krb5kdc[14056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405710, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for kudu/127.0.191.253@KRBTEST.COM
I20260926 06:55:11.159920 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.153341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58138 (local address 127.0.191.253:35833)
0926 06:55:11.153411 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.153413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.153424 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:55:11.153493 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.153495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.153507 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.153578 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.153907 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.154411 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.155175 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.155405 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.155515 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.157526 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.157527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.157528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.157535 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.158818 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.159156 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.159157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.159158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.159215 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.159417 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.159418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.159418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.159527 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.159598 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.159758 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.159833 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 06:55:11.159934 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.153291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:35833 (local address 127.0.0.1:58138)
0926 06:55:11.153340 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.153350 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:11.153427 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:11.153584 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:11.153585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:11.153749 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:11.153813 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.153815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.154501 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.154502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.155097 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.155099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.155516 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.155517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.155603 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.156132 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:11.156137 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:11.157434 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:11.158908 (+ 1474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.158909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.158909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.159082 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.159284 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.159284 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.159285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.159345 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.159603 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:11.159604 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:11.159651 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 06:55:11.159734 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 06:55:11.159832 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 06:55:11.160323 14137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:55:11.160667 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.160572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:37693 (local address 127.0.0.1:35246)
0926 06:55:11.160619 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.160644 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 06:55:11.161042 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.160986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:37693 (local address 127.0.0.1:35252)
0926 06:55:11.161010 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.161030 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260926 06:55:11.161137 14137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:37693: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
W20260926 06:55:11.188489 14219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:11.188681 14219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:11.188702 14219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:11.189751 14219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:55:11.189783 14219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:11.189797 14219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:11.189810 14219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:55:11.189821 14219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:55:11.191213 14219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:45989,127.0.191.253:35833,127.0.191.252:37693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.252.keytab
--principal=kudu/127.0.191.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.252:37693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.191.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:11.191401 14219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:11.191560 14219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:55:11.192269 14137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } attempt: 1
I20260926 06:55:11.192737 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.192614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:37693 (local address 127.0.0.1:35256)
0926 06:55:11.192660 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.192698 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260926 06:55:11.193152 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.193050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:37693 (local address 127.0.0.1:35260)
0926 06:55:11.193110 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.193136 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 06:55:11.193269 14137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:37693: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
W20260926 06:55:11.196400 14224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:11.196460 14227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:11.196460 14225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:11.196619 14219 server_base.cc:1034] running on GCE node
I20260926 06:55:11.196866 14219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:11.197148 14219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:11.198307 14219 hybrid_clock.cc:648] HybridClock initialized: now 1790405711198304 us; error 36 us; skew 500 ppm
Sep 26 06:55:11 dist-test-slave-ws74 krb5kdc[14056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405711, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:11.200781 14219 init.cc:377] Logged in from keytab as kudu/127.0.191.252@KRBTEST.COM (short username kudu)
I20260926 06:55:11.201224 14219 webserver.cc:460] Webserver started at http://127.0.191.252:33861/ using document root <none> and password file <none>
I20260926 06:55:11.201501 14219 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:11.201570 14219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:11.201720 14219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:55:11.202760 14219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/data/instance:
uuid: "d033606ddfff4aa5869c4789d7e48d7d"
format_stamp: "Formatted at 2026-09-26 06:55:11 on dist-test-slave-ws74"
server_key: "d74db8dcdfd2e31595ddea93228f7936"
server_key_iv: "aaefc1229006247bc09d7020a9d25bec"
server_key_version: "encryptionkey@0"
I20260926 06:55:11.203127 14219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/wal/instance:
uuid: "d033606ddfff4aa5869c4789d7e48d7d"
format_stamp: "Formatted at 2026-09-26 06:55:11 on dist-test-slave-ws74"
server_key: "d74db8dcdfd2e31595ddea93228f7936"
server_key_iv: "aaefc1229006247bc09d7020a9d25bec"
server_key_version: "encryptionkey@0"
I20260926 06:55:11.204418 14219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:55:11.205128 14234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:11.205293 14219 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:55:11.205408 14219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/wal
uuid: "d033606ddfff4aa5869c4789d7e48d7d"
format_stamp: "Formatted at 2026-09-26 06:55:11 on dist-test-slave-ws74"
server_key: "d74db8dcdfd2e31595ddea93228f7936"
server_key_iv: "aaefc1229006247bc09d7020a9d25bec"
server_key_version: "encryptionkey@0"
I20260926 06:55:11.205490 14219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:11.215528 14219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:11.217280 14219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:11.217499 14219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:11.220134 14212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } attempt: 2
I20260926 06:55:11.220557 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.220434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:37693 (local address 127.0.0.1:35272)
0926 06:55:11.220494 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.220530 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 06:55:11.220955 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.220802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:37693 (local address 127.0.0.1:35282)
0926 06:55:11.220873 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.220915 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 06:55:11.221076 14212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:37693: Network error: Client connection negotiation failed: client connection to 127.0.191.252:37693: connect: Connection refused (error 111)
I20260926 06:55:11.221719 14219 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.252:37693
I20260926 06:55:11.221773 14286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.252:37693 every 8 connection(s)
I20260926 06:55:11.222124 14219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/data/info.pb
I20260926 06:55:11.222927 14287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:11.224435 14287 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:55:11 dist-test-slave-ws74 krb5kdc[14056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405711, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.252@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:11.232298 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 14219
I20260926 06:55:11.232429 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790405559478510-767-0/minicluster-data/master-2/wal/instance
I20260926 06:55:11.232630 767 external_mini_cluster.cc:1428] Setting key fd6792f6f5f8c93fbff7c0b908a5531c
I20260926 06:55:11.233516 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.225105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58832 (local address 127.0.191.254:45989)
0926 06:55:11.225162 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.225163 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.225354 (+ 191us) server_negotiation.cc:407] Connection header received
0926 06:55:11.225852 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.225853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.225862 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.225924 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.226499 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.226957 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.227952 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.228141 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.228191 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.230186 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.230188 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.230188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.230194 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.232040 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.232410 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.232412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.232412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.232496 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.232794 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.232795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.232795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.232946 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.233016 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.233339 (+ 323us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.233444 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 06:55:11.233752 14289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.225072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:45989 (local address 127.0.0.1:58832)
0926 06:55:11.225219 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.225266 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:11.225770 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:11.225987 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:11.225990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:11.226222 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:11.226429 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.226433 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.227059 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.227060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.227877 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.227878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.228228 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.228229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.228341 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.228764 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:11.228772 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:11.230102 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:11.232112 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.232116 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.232118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.232325 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.232577 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.232578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.232578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.232726 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.233026 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:11.233030 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:11.233195 (+ 165us) client_negotiation.cc:769] Sending connection context
0926 06:55:11.233284 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 06:55:11.233508 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":4,"thread_start_us":76,"threads_started":1}
I20260926 06:55:11.234347 14287 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:55:11 dist-test-slave-ws74 krb5kdc[14056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405711, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.252@KRBTEST.COM for kudu/127.0.191.253@KRBTEST.COM
Sep 26 06:55:11 dist-test-slave-ws74 krb5kdc[14056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.252@KRBTEST.COM
I20260926 06:55:11.240962 14289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.234573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:35833 (local address 127.0.0.1:58144)
0926 06:55:11.234624 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.234635 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:11.234673 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:11.234807 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:11.234808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:11.234944 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:11.234987 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.234989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.235616 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.235617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.236223 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.236226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.236678 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.236679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.236746 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.237051 (+ 305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:11.237057 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:11.238310 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:11.240086 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.240087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.240087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.240241 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.240442 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.240442 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.240443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.240490 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.240766 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:11.240766 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:11.240798 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 06:55:11.240847 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 06:55:11.240904 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 06:55:11.241076 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.234663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58144 (local address 127.0.191.253:35833)
0926 06:55:11.234707 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.234709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.234716 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:55:11.234739 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.234740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.234748 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.234804 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.235106 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.235544 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.236296 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.236605 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.236668 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.238400 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.238402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.238402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.238410 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.240006 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.240313 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.240314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.240314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.240373 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.240571 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.240572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.240573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.240710 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.240779 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.240901 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.240979 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:55:11.241600 14287 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:55:11.241966 14291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.233009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35286 (local address 127.0.191.252:37693)
0926 06:55:11.233163 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.233168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.233193 (+ 25us) server_negotiation.cc:407] Connection header received
0926 06:55:11.233232 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.233235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.233248 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.233320 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.234071 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.234963 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.235774 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.236049 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.236100 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.239048 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.239054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.239055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.239062 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.240649 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.241036 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.241039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.241039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.241104 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.241335 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.241336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.241336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.241535 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.241694 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.241747 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.241838 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
Sep 26 06:55:11 dist-test-slave-ws74 krb5kdc[14056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405711, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.252@KRBTEST.COM for kudu/127.0.191.252@KRBTEST.COM
Sep 26 06:55:11 dist-test-slave-ws74 krb5kdc[14056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:11.249256 14289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.241847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:37693 (local address 127.0.0.1:35292)
0926 06:55:11.241974 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.241985 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:11.242019 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:11.242133 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:11.242134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:11.242269 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:11.242297 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.242298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.243072 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.243073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.243706 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.243708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.244158 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.244159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.244210 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.244548 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:11.244552 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:11.245909 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:11.248250 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.248250 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.248251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.248417 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.248623 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.248624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.248624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.248677 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.248920 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:11.248921 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:11.248957 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 06:55:11.249024 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 06:55:11.249156 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260926 06:55:11.249269 14292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.241893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35292 (local address 127.0.191.252:37693)
0926 06:55:11.242038 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.242040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.242046 (+ 6us) server_negotiation.cc:407] Connection header received
0926 06:55:11.242064 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.242065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.242074 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.242134 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.242394 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.242997 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.243778 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.244094 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.244140 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.246003 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.246004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.246005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.246011 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.248159 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.248499 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.248500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.248500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.248557 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.248746 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.248747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.248747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.248851 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.248930 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.249049 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.249170 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260926 06:55:11.249977 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.242753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58846 (local address 127.0.191.254:45989)
0926 06:55:11.242793 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.242795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.242802 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:55:11.242824 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.242825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.242834 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.242888 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.243439 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.243957 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.244868 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.245105 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.245177 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.247155 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.247156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.247157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.247163 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.248613 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.248959 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.248960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.248961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.249020 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.249222 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.249223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.249224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.249375 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.249624 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.249814 (+ 190us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.249886 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:55:11.250895 14287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d: Bootstrap starting.
I20260926 06:55:11.251653 14287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:11.251904 14287 log.cc:824] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:11.252664 14287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d: No bootstrap required, opened a new log
I20260926 06:55:11.254935 14287 raft_consensus.cc:348] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } }
Sep 26 06:55:11 dist-test-slave-ws74 krb5kdc[14056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.253@KRBTEST.COM
I20260926 06:55:11.255081 14287 raft_consensus.cc:374] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:11.255247 14287 raft_consensus.cc:729] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d033606ddfff4aa5869c4789d7e48d7d, State: Initialized, Role: FOLLOWER
I20260926 06:55:11.255385 14287 consensus_queue.cc:260] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } }
I20260926 06:55:11.255740 14287 sys_catalog.cc:564] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:55:11.257092 14294 sys_catalog.cc:455] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } } }
I20260926 06:55:11.257678 14294 sys_catalog.cc:458] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [sys.catalog]: This master's current role is: FOLLOWER
W20260926 06:55:11.257694 14305 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:55:11.257766 14305 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:55:11.258770 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.250660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58156 (local address 127.0.191.253:35833)
0926 06:55:11.250699 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.250701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.250757 (+ 56us) server_negotiation.cc:407] Connection header received
0926 06:55:11.250829 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.250830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.250838 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.250897 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.251461 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.252007 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.252890 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.253067 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.253122 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.255654 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.255655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.255655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.255662 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.257455 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.257941 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.257942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.257942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.257993 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.258219 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.258221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.258221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.258374 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.258559 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.258598 (+ 39us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.258678 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:55:11.260540 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:55:11.258356 14219 master_runner.cc:186] Error getting master registration for 127.0.191.254:45989: OK, instance_id { permanent_uuid: "1d845f9deff5461e90e1959a133042de" instance_seqno: 1790405710924885 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 06:55:11.260819 14219 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 06:55:11.268417 14137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } attempt: 2
I20260926 06:55:11.268693 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.262128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58860 (local address 127.0.191.254:45989)
0926 06:55:11.262167 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.262169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.262332 (+ 163us) server_negotiation.cc:407] Connection header received
0926 06:55:11.262398 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.262399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.262410 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.262465 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.263046 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.263517 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.264213 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.264421 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.264505 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.265837 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.265838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.265838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.265845 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.267522 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.267920 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.267921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.267921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.267977 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.268166 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.268167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.268167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.268277 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.268344 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.268536 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.268613 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:55:11.273923 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.262132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58158 (local address 127.0.191.253:35833)
0926 06:55:11.262165 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.262167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.265581 (+ 3414us) server_negotiation.cc:407] Connection header received
0926 06:55:11.265632 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.265633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.265643 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.265702 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.266638 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.267626 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.268823 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.269401 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.269446 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.270844 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.270845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.270846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.270852 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.272707 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.273061 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.273062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.273062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.273117 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.273315 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.273316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.273316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.273463 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.273556 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.273758 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.273834 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 26 06:55:11 dist-test-slave-ws74 krb5kdc[14056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405710, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for kudu/127.0.191.252@KRBTEST.COM
I20260926 06:55:11.274703 14292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.261994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35304 (local address 127.0.191.252:37693)
0926 06:55:11.262031 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.262033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.265707 (+ 3674us) server_negotiation.cc:407] Connection header received
0926 06:55:11.265799 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.265800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.265809 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.265839 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.266456 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.267166 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.268183 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.268518 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.268560 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.271129 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.271130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.271130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.271136 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.272765 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.273696 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.273697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.273698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.273750 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.273943 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.273944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.273944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.274094 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.274171 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.274534 (+ 363us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.274616 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:55:11.275568 14294 raft_consensus.cc:482] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:55:11.275650 14294 raft_consensus.cc:504] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } }
I20260926 06:55:11.275897 14294 leader_election.cc:288] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d845f9deff5461e90e1959a133042de (127.0.191.254:45989), f37a293f7bc14090b517b1d3bb4a302d (127.0.191.253:35833)
I20260926 06:55:11.277204 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.268850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:37693 (local address 127.0.0.1:35308)
0926 06:55:11.268984 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.268993 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:11.269027 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:11.269186 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:11.269187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:11.269352 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:11.269408 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.269410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.270272 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.270274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.271085 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.271088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.271853 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.271854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.271923 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.272878 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:11.272882 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:11.274316 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:11.276186 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.276187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.276187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.276402 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.276591 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.276592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.276592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.276664 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.276914 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:11.276915 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:11.276956 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 06:55:11.277025 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 06:55:11.277108 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260926 06:55:11.277213 14291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.269004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35308 (local address 127.0.191.252:37693)
0926 06:55:11.269049 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.269052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.269061 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:55:11.269080 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.269080 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.269089 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.269143 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.269490 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.270182 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.271175 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.271508 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.271579 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.274412 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.274413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.274414 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.274420 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.276119 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.276472 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.276473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.276473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.276530 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.276719 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.276720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.276720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.276852 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.276917 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.277054 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.277127 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 06:55:11.278440 14137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de: Bootstrap starting.
I20260926 06:55:11.279098 14137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:11.279381 14137 log.cc:824] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:11.280210 14137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de: No bootstrap required, opened a new log
I20260926 06:55:11.282209 14137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } }
I20260926 06:55:11.282364 14137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:11.282401 14137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d845f9deff5461e90e1959a133042de, State: Initialized, Role: FOLLOWER
I20260926 06:55:11.282557 14137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } }
I20260926 06:55:11.282930 14137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:55:11.282899 14315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } } }
I20260926 06:55:11.282951 14111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d033606ddfff4aa5869c4789d7e48d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d845f9deff5461e90e1959a133042de" is_pre_election: true
I20260926 06:55:11.283104 14315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:55:11.283176 14111 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d033606ddfff4aa5869c4789d7e48d7d in term 0.
I20260926 06:55:11.283403 14237 leader_election.cc:302] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d845f9deff5461e90e1959a133042de, d033606ddfff4aa5869c4789d7e48d7d; no voters:
I20260926 06:55:11.283543 14294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:55:11.283630 14294 raft_consensus.cc:482] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:55:11.283665 14294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:11.284556 14294 raft_consensus.cc:504] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } }
I20260926 06:55:11.284747 14294 leader_election.cc:288] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [CANDIDATE]: Term 1 election: Requested vote from peers 1d845f9deff5461e90e1959a133042de (127.0.191.254:45989), f37a293f7bc14090b517b1d3bb4a302d (127.0.191.253:35833)
I20260926 06:55:11.285109 14111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d033606ddfff4aa5869c4789d7e48d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d845f9deff5461e90e1959a133042de"
I20260926 06:55:11.285172 14111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:11.286573 14111 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d033606ddfff4aa5869c4789d7e48d7d in term 1.
I20260926 06:55:11.286751 14237 leader_election.cc:302] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d845f9deff5461e90e1959a133042de, d033606ddfff4aa5869c4789d7e48d7d; no voters:
I20260926 06:55:11.286890 14294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:55:11.287025 14294 raft_consensus.cc:686] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [term 1 LEADER]: Becoming Leader. State: Replica: d033606ddfff4aa5869c4789d7e48d7d, State: Running, Role: LEADER
I20260926 06:55:11.287107 14294 consensus_queue.cc:237] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } }
I20260926 06:55:11.287467 14294 sys_catalog.cc:455] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d033606ddfff4aa5869c4789d7e48d7d. Latest consensus state: current_term: 1 leader_uuid: "d033606ddfff4aa5869c4789d7e48d7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } } }
I20260926 06:55:11.287534 14294 sys_catalog.cc:458] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [sys.catalog]: This master's current role is: LEADER
W20260926 06:55:11.284741 14326 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:55:11.287783 14326 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:55:11.287559 14069 master_runner.cc:186] Error getting master registration for 127.0.191.253:35833: OK, instance_id { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" instance_seqno: 1790405711078259 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 06:55:11.289796 14069 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 06:55:11.293958 14330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:55:11.294165 14330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:55:11.295655 14111 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [term 1 FOLLOWER]: Refusing update from remote peer d033606ddfff4aa5869c4789d7e48d7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:55:11.295989 14294 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:55:11.297215 14315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [sys.catalog]: SysCatalogTable state changed. Reason: New leader d033606ddfff4aa5869c4789d7e48d7d. Latest consensus state: current_term: 1 leader_uuid: "d033606ddfff4aa5869c4789d7e48d7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } } }
I20260926 06:55:11.297317 14315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:55:11.298290 14294 sys_catalog.cc:455] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d033606ddfff4aa5869c4789d7e48d7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } } }
I20260926 06:55:11.298386 14294 sys_catalog.cc:458] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:11.299348 14315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d033606ddfff4aa5869c4789d7e48d7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } } }
I20260926 06:55:11.299422 14315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:55:11.302354 14330 catalog_manager.cc:1347] Generated new cluster ID: 7919d703ce9a4d69aca95e87bbe7c856
I20260926 06:55:11.302418 14330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:55:11.312223 14212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } attempt: 3
Sep 26 06:55:11 dist-test-slave-ws74 krb5kdc[14056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405711, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.253@KRBTEST.COM for kudu/127.0.191.252@KRBTEST.COM
I20260926 06:55:11.321233 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.312503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:37693 (local address 127.0.0.1:35320)
0926 06:55:11.312580 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:11.312591 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:11.312628 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:11.312876 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:11.312876 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:11.313064 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:11.313112 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.313114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.314079 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.314080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.314898 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:11.314900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.315422 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.315424 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:11.315479 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.315977 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:11.315982 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:11.317551 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:11.320043 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.320043 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.320044 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.320260 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.320524 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:11.320525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:11.320525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:11.320597 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:11.320922 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:11.320923 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:11.320982 (+ 59us) client_negotiation.cc:769] Sending connection context
0926 06:55:11.321041 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 06:55:11.321132 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 06:55:11.321652 14291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:11.312714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35320 (local address 127.0.191.252:37693)
0926 06:55:11.312750 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:11.312752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:11.312759 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:55:11.312778 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:11.312779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:11.312788 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:11.312832 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:11.313191 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.313987 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.314992 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:11.315301 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:11.315369 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:11.317642 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:11.317643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:11.317643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:11.317650 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:11.319931 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.320364 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.320365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.320366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.320437 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:11.320679 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:11.320680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:11.320680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:11.320818 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:11.320875 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:11.321476 (+ 601us) server_negotiation.cc:299] Negotiation successful
0926 06:55:11.321555 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":19}
I20260926 06:55:11.323094 14330 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:55:11.323422 14212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d: Bootstrap starting.
I20260926 06:55:11.323796 14330 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:55:11.324139 14212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:11.324457 14212 log.cc:824] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:11.325532 14212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d: No bootstrap required, opened a new log
I20260926 06:55:11.327499 14212 raft_consensus.cc:348] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } }
I20260926 06:55:11.327646 14212 raft_consensus.cc:374] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:11.327687 14212 raft_consensus.cc:729] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f37a293f7bc14090b517b1d3bb4a302d, State: Initialized, Role: FOLLOWER
I20260926 06:55:11.327800 14212 consensus_queue.cc:260] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } }
I20260926 06:55:11.328088 14212 sys_catalog.cc:564] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:55:11.329351 14185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d033606ddfff4aa5869c4789d7e48d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f37a293f7bc14090b517b1d3bb4a302d"
I20260926 06:55:11.329478 14185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:11.330271 14185 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d033606ddfff4aa5869c4789d7e48d7d in term 1.
I20260926 06:55:11.330471 14184 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [term 1 FOLLOWER]: Refusing update from remote peer d033606ddfff4aa5869c4789d7e48d7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:55:11.330595 14186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d033606ddfff4aa5869c4789d7e48d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f37a293f7bc14090b517b1d3bb4a302d" is_pre_election: true
I20260926 06:55:11.330654 14186 raft_consensus.cc:2362] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d033606ddfff4aa5869c4789d7e48d7d in term 1. Re-sending same reply.
W20260926 06:55:11.330991 14165 tablet.cc:2369] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:55:11.331866 14294 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:55:11.333935 14337 sys_catalog.cc:455] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d033606ddfff4aa5869c4789d7e48d7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } } }
I20260926 06:55:11.334043 14337 sys_catalog.cc:458] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:55:11.334102 14337 sys_catalog.cc:455] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d033606ddfff4aa5869c4789d7e48d7d. Latest consensus state: current_term: 1 leader_uuid: "d033606ddfff4aa5869c4789d7e48d7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } } }
I20260926 06:55:11.334156 14337 sys_catalog.cc:458] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:55:11.334189 14337 sys_catalog.cc:455] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d033606ddfff4aa5869c4789d7e48d7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } } }
I20260926 06:55:11.334245 14337 sys_catalog.cc:458] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d [sys.catalog]: This master's current role is: FOLLOWER
W20260926 06:55:11.334851 14348 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:55:11.334908 14348 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:55:11.340342 14330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d: Generated new TSK 0
I20260926 06:55:11.340523 14330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:55:11.341599 14350 mvcc.cc:205] Tried to move back new op lower bound from 7333501793574367232 to 7333501793432670208. Current Snapshot: MvccSnapshot[applied={T|T < 7333501793464725504}]
I20260926 06:55:11.341980 14328 sys_catalog.cc:455] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d033606ddfff4aa5869c4789d7e48d7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d845f9deff5461e90e1959a133042de" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 45989 } } peers { permanent_uuid: "f37a293f7bc14090b517b1d3bb4a302d" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 35833 } } peers { permanent_uuid: "d033606ddfff4aa5869c4789d7e48d7d" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 37693 } } }
I20260926 06:55:11.342095 14328 sys_catalog.cc:458] T 00000000000000000000000000000000 P d033606ddfff4aa5869c4789d7e48d7d [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:12.288251 14326 catalog_manager.cc:1259] Loaded cluster ID: 7919d703ce9a4d69aca95e87bbe7c856
I20260926 06:55:12.288326 14326 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de: loading cluster ID for follower catalog manager: success
I20260926 06:55:12.289073 14326 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de: acquiring CA information for follower catalog manager: success
I20260926 06:55:12.289317 14326 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1d845f9deff5461e90e1959a133042de: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:55:12.335373 14348 catalog_manager.cc:1259] Loaded cluster ID: 7919d703ce9a4d69aca95e87bbe7c856
I20260926 06:55:12.335454 14348 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d: loading cluster ID for follower catalog manager: success
I20260926 06:55:12.336280 14348 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d: acquiring CA information for follower catalog manager: success
I20260926 06:55:12.336575 14348 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f37a293f7bc14090b517b1d3bb4a302d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:55:12.396692 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 14069
I20260926 06:55:12.401461 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 14144
I20260926 06:55:12.406211 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 14219
2026-09-26T06:55:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2762 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:55:12 dist-test-slave-ws74 krb5kdc[14363](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:55:12 dist-test-slave-ws74 krb5kdc[14363](info): set up 2 sockets
Sep 26 06:55:12 dist-test-slave-ws74 krb5kdc[14363](info): commencing operation
krb5kdc: starting...
W20260926 06:55:13.437608 767 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:55:13 dist-test-slave-ws74 krb5kdc[14363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:55:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:55:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:55:13.557936 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:42561
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:38681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:42561
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:13.637804 14376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:13.638008 14376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:13.638031 14376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:13.639186 14376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:55:13.639231 14376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:13.639246 14376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:13.639259 14376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:55:13.639310 14376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:55:13.640774 14376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:38681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:42561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:42561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:13.641039 14376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:13.641278 14376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:13.646184 14381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:13.646238 14382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:13.646184 14384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:13.646587 14376 server_base.cc:1034] running on GCE node
I20260926 06:55:13.646881 14376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:13.647177 14376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:13.648345 14376 hybrid_clock.cc:648] HybridClock initialized: now 1790405713648321 us; error 42 us; skew 500 ppm
Sep 26 06:55:13 dist-test-slave-ws74 krb5kdc[14363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405713, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:13.650815 14376 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:55:13.651319 14376 webserver.cc:460] Webserver started at http://127.0.191.254:40901/ using document root <none> and password file <none>
I20260926 06:55:13.651554 14376 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:13.651623 14376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:13.651772 14376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:55:13.652880 14376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "0a69ae864b794fe2960027a48d238f85"
format_stamp: "Formatted at 2026-09-26 06:55:13 on dist-test-slave-ws74"
server_key: "6d7ac30dc027f3b7f01b6d9459a97da6"
server_key_iv: "4d23c96523beb110a0a7e6ca2f6c2f89"
server_key_version: "encryptionkey@0"
I20260926 06:55:13.653247 14376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "0a69ae864b794fe2960027a48d238f85"
format_stamp: "Formatted at 2026-09-26 06:55:13 on dist-test-slave-ws74"
server_key: "6d7ac30dc027f3b7f01b6d9459a97da6"
server_key_iv: "4d23c96523beb110a0a7e6ca2f6c2f89"
server_key_version: "encryptionkey@0"
I20260926 06:55:13.654577 14376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:55:13.655268 14391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:13.655444 14376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:55:13.655540 14376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "0a69ae864b794fe2960027a48d238f85"
format_stamp: "Formatted at 2026-09-26 06:55:13 on dist-test-slave-ws74"
server_key: "6d7ac30dc027f3b7f01b6d9459a97da6"
server_key_iv: "4d23c96523beb110a0a7e6ca2f6c2f89"
server_key_version: "encryptionkey@0"
I20260926 06:55:13.655632 14376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:13.677685 14376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:13.679618 14376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:13.679781 14376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:13.683949 14376 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:42561
I20260926 06:55:13.683983 14443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:42561 every 8 connection(s)
I20260926 06:55:13.684336 14376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:55:13.685185 14444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:13.687479 14444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85: Bootstrap starting.
I20260926 06:55:13.687996 14444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:13.688231 14444 log.cc:824] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:13.688861 14444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85: No bootstrap required, opened a new log
I20260926 06:55:13.690511 14444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a69ae864b794fe2960027a48d238f85" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42561 } }
I20260926 06:55:13.690649 14444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:13.690670 14444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a69ae864b794fe2960027a48d238f85, State: Initialized, Role: FOLLOWER
I20260926 06:55:13.690768 14444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a69ae864b794fe2960027a48d238f85" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42561 } }
I20260926 06:55:13.690816 14444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:55:13.690836 14444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:55:13.690867 14444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:13.691340 14444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a69ae864b794fe2960027a48d238f85" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42561 } }
I20260926 06:55:13.691419 14444 leader_election.cc:302] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a69ae864b794fe2960027a48d238f85; no voters:
I20260926 06:55:13.691573 14444 leader_election.cc:288] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:55:13.691602 14447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:55:13.691722 14447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [term 1 LEADER]: Becoming Leader. State: Replica: 0a69ae864b794fe2960027a48d238f85, State: Running, Role: LEADER
I20260926 06:55:13.691838 14444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:55:13.691890 14447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a69ae864b794fe2960027a48d238f85" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42561 } }
I20260926 06:55:13.692342 14448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a69ae864b794fe2960027a48d238f85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a69ae864b794fe2960027a48d238f85" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42561 } } }
I20260926 06:55:13.692446 14448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:13.692466 14449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a69ae864b794fe2960027a48d238f85. Latest consensus state: current_term: 1 leader_uuid: "0a69ae864b794fe2960027a48d238f85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a69ae864b794fe2960027a48d238f85" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42561 } } }
I20260926 06:55:13.692553 14449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85 [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:13.692605 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 14376
I20260926 06:55:13.692696 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:55:13.692900 767 external_mini_cluster.cc:1428] Setting key 4750e927ea0dd99dda3147be7383578c
I20260926 06:55:13.692809 14456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:55:13.693435 14456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:55:13.695053 14456 catalog_manager.cc:1347] Generated new cluster ID: ab5ef6781fd840dfa7e6032650833e6e
I20260926 06:55:13.695106 14456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:55:13 dist-test-slave-ws74 krb5kdc[14363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:13.703599 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:13.693472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51284 (local address 127.0.191.254:42561)
0926 06:55:13.694197 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:13.694206 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:13.694238 (+ 32us) server_negotiation.cc:407] Connection header received
0926 06:55:13.694351 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:13.694357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:13.694412 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:13.694589 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:13.695321 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:13.696214 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:13.696960 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:13.697425 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:13.697524 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:13.699910 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:13.699915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:13.699915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:13.699931 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:13.702079 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:13.702456 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:13.702458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:13.702459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:13.702557 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:13.702779 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:13.702780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:13.702781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:13.703006 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:13.703187 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:13.703257 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 06:55:13.703409 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":70,"threads_started":1}
I20260926 06:55:13.704475 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:55:13.707521 14456 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:55:13.708039 14456 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:55:13.712536 14456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a69ae864b794fe2960027a48d238f85: Generated new TSK 0
I20260926 06:55:13.712702 14456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:55:13.758222 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 14376
2026-09-26T06:55:13Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1350 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:55:13 dist-test-slave-ws74 krb5kdc[14471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:55:13 dist-test-slave-ws74 krb5kdc[14471](info): set up 2 sockets
Sep 26 06:55:13 dist-test-slave-ws74 krb5kdc[14471](info): commencing operation
krb5kdc: starting...
W20260926 06:55:14.793570 767 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:55:14 dist-test-slave-ws74 krb5kdc[14471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:55:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:55:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:55:14.912135 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:39441
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:39441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:14.991942 14484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:14.992129 14484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:14.992149 14484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:14.993296 14484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:55:14.993346 14484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:14.993360 14484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:14.993420 14484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:55:14.993445 14484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:55:14.994892 14484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:39441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:39441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:14.995157 14484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:14.995397 14484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:15.000135 14492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:15.000240 14490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:15.000311 14484 server_base.cc:1034] running on GCE node
W20260926 06:55:15.000136 14489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:15.000669 14484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:15.000958 14484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:15.002118 14484 hybrid_clock.cc:648] HybridClock initialized: now 1790405715002107 us; error 27 us; skew 500 ppm
Sep 26 06:55:14 dist-test-slave-ws74 krb5kdc[14471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405714, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:15.004551 14484 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:55:15.004961 14484 webserver.cc:460] Webserver started at http://127.0.191.254:39995/ using document root <none> and password file <none>
I20260926 06:55:15.005196 14484 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:15.005265 14484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:15.005446 14484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:55:15.006493 14484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "26ae31482d134307a6ea0000774f460d"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "de8752365b5f639e41b7348e6d8a88e3"
server_key_iv: "bd6ac8ae324b08edc245d451aa61c2de"
server_key_version: "encryptionkey@0"
I20260926 06:55:15.006840 14484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "26ae31482d134307a6ea0000774f460d"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "de8752365b5f639e41b7348e6d8a88e3"
server_key_iv: "bd6ac8ae324b08edc245d451aa61c2de"
server_key_version: "encryptionkey@0"
I20260926 06:55:15.008095 14484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:55:15.008796 14499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:15.008980 14484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:55:15.009078 14484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "26ae31482d134307a6ea0000774f460d"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "de8752365b5f639e41b7348e6d8a88e3"
server_key_iv: "bd6ac8ae324b08edc245d451aa61c2de"
server_key_version: "encryptionkey@0"
I20260926 06:55:15.009162 14484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:15.026486 14484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:15.028867 14484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:15.029036 14484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:15.033432 14484 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:39441
I20260926 06:55:15.033481 14551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:39441 every 8 connection(s)
I20260926 06:55:15.033854 14484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:55:15.034842 14552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:15.036778 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 14484
I20260926 06:55:15.036928 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:55:15.037158 767 external_mini_cluster.cc:1428] Setting key f4ad781c717549b46b9d1ea447a0a2c9
I20260926 06:55:15.037185 14552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Bootstrap starting.
I20260926 06:55:15.037861 14552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:15.038149 14552 log.cc:824] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:15.038851 14552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: No bootstrap required, opened a new log
I20260926 06:55:15.040606 14552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } }
I20260926 06:55:15.040759 14552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:15.040799 14552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26ae31482d134307a6ea0000774f460d, State: Initialized, Role: FOLLOWER
I20260926 06:55:15.040975 14552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } }
I20260926 06:55:15.041069 14552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:55:15.041107 14552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:55:15.041157 14552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:15.041692 14552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } }
I20260926 06:55:15.041810 14552 leader_election.cc:302] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26ae31482d134307a6ea0000774f460d; no voters:
I20260926 06:55:15.042006 14552 leader_election.cc:288] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:55:15.042027 14557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:55:15.042136 14557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 1 LEADER]: Becoming Leader. State: Replica: 26ae31482d134307a6ea0000774f460d, State: Running, Role: LEADER
I20260926 06:55:15.042214 14557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } }
I20260926 06:55:15.042269 14552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:55:15.042521 14557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26ae31482d134307a6ea0000774f460d. Latest consensus state: current_term: 1 leader_uuid: "26ae31482d134307a6ea0000774f460d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } } }
I20260926 06:55:15.042615 14557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:15.042536 14558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26ae31482d134307a6ea0000774f460d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } } }
I20260926 06:55:15.042660 14558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: This master's current role is: LEADER
W20260926 06:55:15.044024 14573 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:55:15.044413 14573 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:55:15.044497 14561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:55:15.044677 14561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 06:55:15 dist-test-slave-ws74 krb5kdc[14471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:15.046777 14561 catalog_manager.cc:1347] Generated new cluster ID: 76adea3907524256bde19621f97fb390
I20260926 06:55:15.046833 14561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:55:15.048821 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.037740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58028 (local address 127.0.191.254:39441)
0926 06:55:15.037964 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:15.037974 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:15.038012 (+ 38us) server_negotiation.cc:407] Connection header received
0926 06:55:15.038634 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:15.038643 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:15.038715 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:15.038873 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:15.039574 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.040518 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.041641 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.042263 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.043724 (+ 1461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.045489 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:15.045493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:15.045494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:15.045506 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:15.047374 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:15.047732 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:15.047735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:15.047735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:15.047826 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:15.048032 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:15.048033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:15.048034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:15.048252 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:15.048418 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:15.048527 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 06:55:15.048660 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":93,"threads_started":1}
I20260926 06:55:15.053359 14561 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:55:15.054152 14561 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:55:15.062387 14561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Generated new TSK 0
I20260926 06:55:15.062572 14561 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:55:15.096436 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:39441
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:15.178742 14578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:15.178960 14578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:15.178985 14578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:15.180234 14578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:15.180279 14578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:15.180310 14578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:55:15.181962 14578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:39441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:15.182237 14578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:15.182492 14578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:15.188079 14584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:15.188095 14583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:15.188079 14586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:15.188340 14578 server_base.cc:1034] running on GCE node
I20260926 06:55:15.188527 14578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:15.188714 14578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:15.189842 14578 hybrid_clock.cc:648] HybridClock initialized: now 1790405715189828 us; error 31 us; skew 500 ppm
Sep 26 06:55:15 dist-test-slave-ws74 krb5kdc[14471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:15.192276 14578 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:55:15.192646 14578 webserver.cc:460] Webserver started at http://127.0.191.193:35095/ using document root <none> and password file <none>
I20260926 06:55:15.192833 14578 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:15.192873 14578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:15.192957 14578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:55:15.194001 14578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "aa0e00f504e64287b73bf26a8f445232"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "c08be58152c56ab04f110befc07bd1f5"
server_key_iv: "caf37b5f3bb9a3cdc7f8cec7a0412e91"
server_key_version: "encryptionkey@0"
I20260926 06:55:15.194300 14578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "aa0e00f504e64287b73bf26a8f445232"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "c08be58152c56ab04f110befc07bd1f5"
server_key_iv: "caf37b5f3bb9a3cdc7f8cec7a0412e91"
server_key_version: "encryptionkey@0"
I20260926 06:55:15.195526 14578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:55:15.196210 14593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:15.196376 14578 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:55:15.196477 14578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "aa0e00f504e64287b73bf26a8f445232"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "c08be58152c56ab04f110befc07bd1f5"
server_key_iv: "caf37b5f3bb9a3cdc7f8cec7a0412e91"
server_key_version: "encryptionkey@0"
I20260926 06:55:15.196557 14578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:15.210886 14578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:15.213050 14578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:15.213214 14578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:15.213469 14578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:15.213786 14578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:15.213841 14578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:15.213881 14578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:15.213910 14578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:15.219406 14578 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:39687
I20260926 06:55:15.219481 14706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:39687 every 8 connection(s)
I20260926 06:55:15.219802 14578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:55:15.221164 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 14578
I20260926 06:55:15.221274 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:55:15.221529 767 external_mini_cluster.cc:1428] Setting key eaa1cfab78ef409a653b21c5ea51fbdf
Sep 26 06:55:15 dist-test-slave-ws74 krb5kdc[14471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:15.229774 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.220503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:60859 (local address 127.0.191.254:39441)
0926 06:55:15.220583 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:15.220585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:15.220767 (+ 182us) server_negotiation.cc:407] Connection header received
0926 06:55:15.221411 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:15.221412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:15.221425 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:15.221490 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:15.222614 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.223097 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.224236 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.224587 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.224661 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.226767 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:15.226769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:15.226769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:15.226778 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:15.228362 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:15.228733 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:15.228734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:15.228735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:15.228790 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:15.229012 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:15.229013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:15.229013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:15.229157 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:15.229233 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:15.229566 (+ 333us) server_negotiation.cc:299] Negotiation successful
0926 06:55:15.229665 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 06:55:15.229791 14709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.220464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:60859)
0926 06:55:15.220636 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:15.220671 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:15.221291 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:15.221568 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:15.221572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:15.222214 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:15.222522 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.222528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.223201 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.223202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.224143 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.224145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.224674 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.224675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.224789 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.225299 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:15.225307 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:15.226678 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:15.228449 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:15.228455 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:15.228456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:15.228659 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:15.228857 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:15.228857 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:15.228858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:15.228947 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:15.229228 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:15.229231 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:15.229364 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 06:55:15.229519 (+ 155us) client_negotiation.cc:240] Negotiation successful
0926 06:55:15.229632 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
I20260926 06:55:15.230362 14707 heartbeater.cc:347] Connected to a master server at 127.0.191.254:39441
I20260926 06:55:15.230466 14707 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:15.230651 14707 heartbeater.cc:511] Master 127.0.191.254:39441 requested a full tablet report, sending...
I20260926 06:55:15.231127 14516 ts_manager.cc:194] Registered new tserver with Master: aa0e00f504e64287b73bf26a8f445232 (127.0.191.193:39687)
I20260926 06:55:15.231766 14516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:60859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:55:15.248059 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:39441
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:15.327844 14712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:15.328044 14712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:15.328068 14712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:15.329129 14712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:15.329161 14712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:15.329186 14712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:55:15.330559 14712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:39441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:15.330734 14712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:15.330905 14712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:15.335793 14718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:15.335901 14720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:15.335974 14712 server_base.cc:1034] running on GCE node
W20260926 06:55:15.335793 14717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:15.336290 14712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:15.336519 14712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:15.337668 14712 hybrid_clock.cc:648] HybridClock initialized: now 1790405715337667 us; error 34 us; skew 500 ppm
Sep 26 06:55:15 dist-test-slave-ws74 krb5kdc[14471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:15.340001 14712 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:55:15.340404 14712 webserver.cc:460] Webserver started at http://127.0.191.194:44529/ using document root <none> and password file <none>
I20260926 06:55:15.340631 14712 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:15.340693 14712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:15.340838 14712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:55:15.341887 14712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "2185dc0f2d344beb82881db37c91d88b"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "d174c0acb79d6c60a43d44e146690883"
server_key_iv: "5a818f67a218b2087fa09443aa5bd8bc"
server_key_version: "encryptionkey@0"
I20260926 06:55:15.342208 14712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "2185dc0f2d344beb82881db37c91d88b"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "d174c0acb79d6c60a43d44e146690883"
server_key_iv: "5a818f67a218b2087fa09443aa5bd8bc"
server_key_version: "encryptionkey@0"
I20260926 06:55:15.343636 14712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:55:15.344355 14727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:15.344529 14712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:55:15.344632 14712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "2185dc0f2d344beb82881db37c91d88b"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "d174c0acb79d6c60a43d44e146690883"
server_key_iv: "5a818f67a218b2087fa09443aa5bd8bc"
server_key_version: "encryptionkey@0"
I20260926 06:55:15.344702 14712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:15.354398 14712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:15.356599 14712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:15.356760 14712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:15.356974 14712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:15.357271 14712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:15.357321 14712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:15.357360 14712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:15.357410 14712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:15.363183 14712 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:42225
I20260926 06:55:15.363261 14840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:42225 every 8 connection(s)
I20260926 06:55:15.363556 14712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:55:15 dist-test-slave-ws74 krb5kdc[14471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:15.372656 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.364300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:46689 (local address 127.0.191.254:39441)
0926 06:55:15.364345 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:15.364346 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:15.364553 (+ 207us) server_negotiation.cc:407] Connection header received
0926 06:55:15.365074 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:15.365075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:15.365088 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:15.365139 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:15.366128 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.366628 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.367636 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.367855 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.367941 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.370132 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:15.370134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:15.370134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:15.370141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:15.371320 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:15.371716 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:15.371734 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:15.371734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:15.371796 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:15.372040 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:15.372041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:15.372041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:15.372151 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:15.372219 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:15.372490 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 06:55:15.372569 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:55:15.372684 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 14712
I20260926 06:55:15.372786 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:55:15.372824 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.364248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:46689)
0926 06:55:15.364441 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:15.364475 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:15.364963 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:15.365204 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:15.365208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:15.365826 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:15.366014 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.366018 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.366734 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.366735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.367531 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.367533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.367952 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.367953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.368089 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.368561 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:15.368569 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:15.370017 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:15.371411 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:15.371417 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:15.371419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:15.371628 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:15.371863 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:15.371864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:15.371865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:15.371967 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:15.372232 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:15.372238 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:15.372360 (+ 122us) client_negotiation.cc:769] Sending connection context
0926 06:55:15.372455 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 06:55:15.372600 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
I20260926 06:55:15.372979 767 external_mini_cluster.cc:1428] Setting key fb5eea869db7464a8e176ecb6c4322a9
I20260926 06:55:15.373291 14841 heartbeater.cc:347] Connected to a master server at 127.0.191.254:39441
I20260926 06:55:15.373405 14841 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:15.373586 14841 heartbeater.cc:511] Master 127.0.191.254:39441 requested a full tablet report, sending...
I20260926 06:55:15.373955 14516 ts_manager.cc:194] Registered new tserver with Master: 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225)
I20260926 06:55:15.374341 14516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:46689
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:55:15.397554 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:39441
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:15.479959 14846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:15.480173 14846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:15.480206 14846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:15.481456 14846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:15.481498 14846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:15.481555 14846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:55:15.482967 14846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:39441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:15.483238 14846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:15.483469 14846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:15.488521 14851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:15.488597 14846 server_base.cc:1034] running on GCE node
W20260926 06:55:15.488615 14852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:15.488787 14854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:15.489006 14846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:15.489204 14846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:15.490357 14846 hybrid_clock.cc:648] HybridClock initialized: now 1790405715490336 us; error 46 us; skew 500 ppm
Sep 26 06:55:15 dist-test-slave-ws74 krb5kdc[14471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:15.492786 14846 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:55:15.493170 14846 webserver.cc:460] Webserver started at http://127.0.191.195:34101/ using document root <none> and password file <none>
I20260926 06:55:15.493451 14846 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:15.493520 14846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:15.493672 14846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:55:15.494690 14846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "90b4d59ee0484dada801834f1680ad58"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "aec471c60d83b2e7b6cafaa2efcdfcc9"
server_key_iv: "942b94a21454449542539410417e425b"
server_key_version: "encryptionkey@0"
I20260926 06:55:15.495055 14846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "90b4d59ee0484dada801834f1680ad58"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "aec471c60d83b2e7b6cafaa2efcdfcc9"
server_key_iv: "942b94a21454449542539410417e425b"
server_key_version: "encryptionkey@0"
I20260926 06:55:15.496356 14846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:55:15.497054 14861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:15.497229 14846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:55:15.497334 14846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "90b4d59ee0484dada801834f1680ad58"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "aec471c60d83b2e7b6cafaa2efcdfcc9"
server_key_iv: "942b94a21454449542539410417e425b"
server_key_version: "encryptionkey@0"
I20260926 06:55:15.497442 14846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:15.528112 14846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:15.530431 14846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:15.530586 14846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:15.530807 14846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:15.531117 14846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:15.531170 14846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:15.531211 14846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:15.531237 14846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:15.537226 14846 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:35645
I20260926 06:55:15.537307 14974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:35645 every 8 connection(s)
I20260926 06:55:15.537616 14846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:55:15.542069 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 14846
I20260926 06:55:15.542202 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:55:15.542629 767 external_mini_cluster.cc:1428] Setting key 84ee5bec27a998cd9ce0d088c5e7d6e3
Sep 26 06:55:15 dist-test-slave-ws74 krb5kdc[14471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:15.546342 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.538271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:53987 (local address 127.0.191.254:39441)
0926 06:55:15.538312 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:15.538314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:15.538518 (+ 204us) server_negotiation.cc:407] Connection header received
0926 06:55:15.538983 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:15.538984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:15.538994 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:15.539045 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:15.539886 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.540368 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.541312 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.541546 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.541598 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.544026 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:15.544026 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:15.544027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:15.544032 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:15.545122 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:15.545509 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:15.545510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:15.545510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:15.545561 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:15.545750 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:15.545750 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:15.545751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:15.545848 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:15.545899 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:15.546166 (+ 267us) server_negotiation.cc:299] Negotiation successful
0926 06:55:15.546253 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:55:15.546516 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.538249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:53987)
0926 06:55:15.538403 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:15.538438 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:15.538904 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:15.539099 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:15.539102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:15.539635 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:15.539815 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.539819 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.540440 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.540440 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.541241 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.541243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.541614 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.541615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.541719 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.542254 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:15.542264 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:15.543939 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:15.545188 (+ 1249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:15.545194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:15.545195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:15.545447 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:15.545606 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:15.545607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:15.545607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:15.545699 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:15.545907 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:15.545912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:15.546016 (+ 104us) client_negotiation.cc:769] Sending connection context
0926 06:55:15.546124 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 06:55:15.546275 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":91,"threads_started":1}
I20260926 06:55:15.547019 14975 heartbeater.cc:347] Connected to a master server at 127.0.191.254:39441
I20260926 06:55:15.547112 14975 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:15.547241 14975 heartbeater.cc:511] Master 127.0.191.254:39441 requested a full tablet report, sending...
I20260926 06:55:15.547583 14516 ts_manager.cc:194] Registered new tserver with Master: 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645)
I20260926 06:55:15.547967 14516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:53987
I20260926 06:55:15.554549 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:55:15.561632 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.555840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58040 (local address 127.0.191.254:39441)
0926 06:55:15.555919 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:15.555921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:15.556014 (+ 93us) server_negotiation.cc:407] Connection header received
0926 06:55:15.556092 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:15.556093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:15.556103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:15.556152 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:15.556700 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.557174 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.557983 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.558177 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.558259 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.559353 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:15.559355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:15.559355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:15.559363 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:15.560435 (+ 1072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:15.560763 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:15.560764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:15.560764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:15.560820 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:15.561032 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:15.561032 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:15.561033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:15.561175 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:15.561286 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:15.561458 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 06:55:15.561534 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 06:55:15.563151 14516 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:55:15.564989 14516 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:55:15.571146 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.567340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49128 (local address 127.0.191.195:35645)
0926 06:55:15.568049 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:15.568053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:15.568063 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:55:15.568100 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:15.568102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:15.568117 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:15.568193 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:15.568400 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.569090 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.570194 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.570766 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.570853 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.570913 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:15.571027 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 06:55:15.571066 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":85,"threads_started":1}
I20260926 06:55:15.571138 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.567227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:35645 (local address 127.0.0.1:49128)
0926 06:55:15.567528 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:15.567563 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:15.567632 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:15.568268 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:15.568269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:15.568271 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:15.568317 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.568318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.569198 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.569199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.570097 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.570099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.570854 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.570854 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.570933 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.570946 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 06:55:15.571015 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 06:55:15.571064 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":4,"spinlock_wait_cycles":384,"thread_start_us":89,"threads_started":1}
I20260926 06:55:15.571556 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.567227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:42225 (local address 127.0.0.1:42744)
0926 06:55:15.567521 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:15.567563 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:15.567632 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:15.567940 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:15.567941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:15.567942 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:15.567992 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.567993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.569044 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.569046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.570327 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.570329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.571283 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.571284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.571350 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.571353 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:55:15.571432 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 06:55:15.571471 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":31,"thread_start_us":46,"threads_started":1}
I20260926 06:55:15.571583 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.567387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42744 (local address 127.0.191.194:42225)
0926 06:55:15.567781 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:15.567785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:15.567794 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:55:15.567833 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:15.567835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:15.567848 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:15.567935 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:15.568130 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.568942 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.570424 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.571122 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.571188 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.571244 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:15.571454 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 06:55:15.571500 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":66,"threads_started":1}
I20260926 06:55:15.571966 14909 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8098587abe49f4b71f26b0a9630035 (DEFAULT_TABLE table=test-table [id=e72538e43cdc4993bcca5d160596e6c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:15.572201 14775 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8098587abe49f4b71f26b0a9630035 (DEFAULT_TABLE table=test-table [id=e72538e43cdc4993bcca5d160596e6c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:15.572314 14909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8098587abe49f4b71f26b0a9630035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:15.572528 14775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8098587abe49f4b71f26b0a9630035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:15.572789 14990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.567420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39687 (local address 127.0.0.1:49442)
0926 06:55:15.567539 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:15.567567 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:15.567631 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:15.567899 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:15.567901 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:15.567903 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:15.567993 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.567994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.570262 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.570263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.571463 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.571465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.572544 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.572545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.572631 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.572636 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:55:15.572689 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 06:55:15.572725 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":4,"thread_start_us":66,"threads_started":1}
I20260926 06:55:15.572830 14991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.567545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49442 (local address 127.0.191.193:39687)
0926 06:55:15.567726 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:15.567732 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:15.567742 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:55:15.567783 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:15.567785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:15.567803 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:15.567886 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:15.569080 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.570153 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.571543 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.572392 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.572459 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.572515 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:15.572703 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 06:55:15.572740 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":73,"threads_started":1}
I20260926 06:55:15.573637 14641 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8098587abe49f4b71f26b0a9630035 (DEFAULT_TABLE table=test-table [id=e72538e43cdc4993bcca5d160596e6c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:15.573985 14641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8098587abe49f4b71f26b0a9630035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:15.575677 14992 tablet_bootstrap.cc:492] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b: Bootstrap starting.
I20260926 06:55:15.575961 14993 tablet_bootstrap.cc:492] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58: Bootstrap starting.
I20260926 06:55:15.576475 14992 tablet_bootstrap.cc:654] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:15.576606 14993 tablet_bootstrap.cc:654] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:15.576720 14994 tablet_bootstrap.cc:492] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Bootstrap starting.
I20260926 06:55:15.577100 14992 log.cc:824] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:15.577190 14993 log.cc:824] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:15.577394 14994 tablet_bootstrap.cc:654] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:15.577907 14994 log.cc:824] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:15.577934 14992 tablet_bootstrap.cc:492] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b: No bootstrap required, opened a new log
I20260926 06:55:15.577946 14993 tablet_bootstrap.cc:492] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58: No bootstrap required, opened a new log
I20260926 06:55:15.578001 14993 ts_tablet_manager.cc:1397] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260926 06:55:15.578001 14992 ts_tablet_manager.cc:1397] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:55:15.578545 14994 tablet_bootstrap.cc:492] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: No bootstrap required, opened a new log
I20260926 06:55:15.578624 14994 ts_tablet_manager.cc:1397] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:55:15.579927 14993 raft_consensus.cc:348] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:15.579911 14992 raft_consensus.cc:348] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:15.580099 14992 raft_consensus.cc:374] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:15.580099 14993 raft_consensus.cc:374] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:15.580135 14992 raft_consensus.cc:729] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2185dc0f2d344beb82881db37c91d88b, State: Initialized, Role: FOLLOWER
I20260926 06:55:15.580137 14993 raft_consensus.cc:729] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90b4d59ee0484dada801834f1680ad58, State: Initialized, Role: FOLLOWER
I20260926 06:55:15.580132 14994 raft_consensus.cc:348] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:15.580251 14992 consensus_queue.cc:260] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:15.580277 14993 consensus_queue.cc:260] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:15.580317 14994 raft_consensus.cc:374] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:15.580365 14994 raft_consensus.cc:729] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0e00f504e64287b73bf26a8f445232, State: Initialized, Role: FOLLOWER
I20260926 06:55:15.580488 14994 consensus_queue.cc:260] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:15.580525 14992 ts_tablet_manager.cc:1428] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 06:55:15.580525 14993 ts_tablet_manager.cc:1428] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 06:55:15.580653 14975 heartbeater.cc:503] Master 127.0.191.254:39441 was elected leader, sending a full tablet report...
I20260926 06:55:15.580667 14841 heartbeater.cc:503] Master 127.0.191.254:39441 was elected leader, sending a full tablet report...
I20260926 06:55:15.580766 14994 ts_tablet_manager.cc:1428] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:55:15.580914 14707 heartbeater.cc:503] Master 127.0.191.254:39441 was elected leader, sending a full tablet report...
W20260926 06:55:15.614071 14842 tablet.cc:2369] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:55:15.720556 14708 tablet.cc:2369] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:55:15.750150 15000 raft_consensus.cc:482] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:55:15.750260 15000 raft_consensus.cc:504] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:15.750639 15000 leader_election.cc:288] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225), 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645)
I20260926 06:55:15.753553 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.750912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:51155 (local address 127.0.191.195:35645)
0926 06:55:15.750956 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:15.750958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:15.751025 (+ 67us) server_negotiation.cc:407] Connection header received
0926 06:55:15.751092 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:15.751093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:15.751108 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:15.751173 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:15.751347 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.751819 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.752777 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.753153 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.753248 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.753257 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:15.753440 (+ 183us) server_negotiation.cc:299] Negotiation successful
0926 06:55:15.753479 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:55:15.753575 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.750860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:34787 (local address 127.0.191.194:42225)
0926 06:55:15.750904 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:15.750907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:15.751022 (+ 115us) server_negotiation.cc:407] Connection header received
0926 06:55:15.751098 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:15.751099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:15.751115 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:15.751177 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:15.751396 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.751856 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.752810 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.753178 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.753274 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.753283 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:15.753466 (+ 183us) server_negotiation.cc:299] Negotiation successful
0926 06:55:15.753506 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:55:15.753598 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.750802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:35645 (local address 127.0.191.193:51155)
0926 06:55:15.750951 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:15.750962 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:15.751026 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:15.751177 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:15.751178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:15.751180 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:15.751258 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.751260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.751905 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.751906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.752684 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.752686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.753248 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.753249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.753327 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.753331 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:55:15.753432 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 06:55:15.753474 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":15,"thread_start_us":91,"threads_started":1}
I20260926 06:55:15.753602 15001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.750795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:42225 (local address 127.0.191.193:34787)
0926 06:55:15.750952 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:15.750961 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:15.751033 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:15.751182 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:15.751190 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:15.751191 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:15.751319 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.751321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.751941 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.751942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.752699 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:15.752701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.753281 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.753281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:15.753354 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.753357 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:55:15.753458 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 06:55:15.753500 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":3,"thread_start_us":89,"threads_started":1}
I20260926 06:55:15.753955 14929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8098587abe49f4b71f26b0a9630035" candidate_uuid: "aa0e00f504e64287b73bf26a8f445232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90b4d59ee0484dada801834f1680ad58" is_pre_election: true
I20260926 06:55:15.754119 14929 raft_consensus.cc:2454] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa0e00f504e64287b73bf26a8f445232 in term 0.
I20260926 06:55:15.754148 14795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8098587abe49f4b71f26b0a9630035" candidate_uuid: "aa0e00f504e64287b73bf26a8f445232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2185dc0f2d344beb82881db37c91d88b" is_pre_election: true
I20260926 06:55:15.754302 14795 raft_consensus.cc:2454] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa0e00f504e64287b73bf26a8f445232 in term 0.
I20260926 06:55:15.754359 14597 leader_election.cc:302] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90b4d59ee0484dada801834f1680ad58, aa0e00f504e64287b73bf26a8f445232; no voters:
I20260926 06:55:15.754468 15000 raft_consensus.cc:2781] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:55:15.754525 15000 raft_consensus.cc:482] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:55:15.754542 15000 raft_consensus.cc:3037] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:15.755348 15000 raft_consensus.cc:504] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:15.755493 15000 leader_election.cc:288] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 1 election: Requested vote from peers 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225), 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645)
I20260926 06:55:15.755604 14795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8098587abe49f4b71f26b0a9630035" candidate_uuid: "aa0e00f504e64287b73bf26a8f445232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2185dc0f2d344beb82881db37c91d88b"
I20260926 06:55:15.755630 14929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8098587abe49f4b71f26b0a9630035" candidate_uuid: "aa0e00f504e64287b73bf26a8f445232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90b4d59ee0484dada801834f1680ad58"
I20260926 06:55:15.755699 14929 raft_consensus.cc:3037] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:15.755703 14795 raft_consensus.cc:3037] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:15.756461 14929 raft_consensus.cc:2454] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa0e00f504e64287b73bf26a8f445232 in term 1.
I20260926 06:55:15.756589 14795 raft_consensus.cc:2454] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa0e00f504e64287b73bf26a8f445232 in term 1.
I20260926 06:55:15.756626 14597 leader_election.cc:302] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90b4d59ee0484dada801834f1680ad58, aa0e00f504e64287b73bf26a8f445232; no voters:
I20260926 06:55:15.756731 15000 raft_consensus.cc:2781] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:55:15.756863 15000 raft_consensus.cc:686] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 1 LEADER]: Becoming Leader. State: Replica: aa0e00f504e64287b73bf26a8f445232, State: Running, Role: LEADER
I20260926 06:55:15.756942 15000 consensus_queue.cc:237] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:15.757794 14515 catalog_manager.cc:5495] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa0e00f504e64287b73bf26a8f445232 (127.0.191.193). New cstate: current_term: 1 leader_uuid: "aa0e00f504e64287b73bf26a8f445232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } health_report { overall_health: HEALTHY } } }
W20260926 06:55:15.788257 14976 tablet.cc:2369] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:55:15.798142 14991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.795478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49446 (local address 127.0.191.193:39687)
0926 06:55:15.795535 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:15.795537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:15.795546 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:55:15.795639 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:15.795640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:15.795657 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:15.795748 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:55:15.795895 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.796361 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.797287 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:15.797542 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:15.797633 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:15.797803 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:55:15.797911 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:55:15.798013 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:15.798038 (+ 25us) server_negotiation.cc:299] Negotiation successful
0926 06:55:15.798059 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:55:15.799954 14929 raft_consensus.cc:1261] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58 [term 1 FOLLOWER]: Refusing update from remote peer aa0e00f504e64287b73bf26a8f445232: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:55:15.799980 14795 raft_consensus.cc:1261] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b [term 1 FOLLOWER]: Refusing update from remote peer aa0e00f504e64287b73bf26a8f445232: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:55:15.800405 15003 consensus_queue.cc:1035] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:55:15.800539 15000 consensus_queue.cc:1035] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:55:15.802954 15008 mvcc.cc:205] Tried to move back new op lower bound from 7333501811913945088 to 7333501811740934144. Current Snapshot: MvccSnapshot[applied={T|T < 7333501811913945088}]
I20260926 06:55:15.803320 15007 mvcc.cc:205] Tried to move back new op lower bound from 7333501811913945088 to 7333501811740934144. Current Snapshot: MvccSnapshot[applied={T|T < 7333501811913945088}]
I20260926 06:55:15.803560 15006 mvcc.cc:205] Tried to move back new op lower bound from 7333501811913945088 to 7333501811740934144. Current Snapshot: MvccSnapshot[applied={T|T < 7333501811913945088 or (T in {7333501811913945088})}]
I20260926 06:55:15.805106 14514 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:55:15.805212 14514 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:55:15.806005 14514 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Sending DeleteTablet for 3 replicas of tablet 4c8098587abe49f4b71f26b0a9630035
I20260926 06:55:15.806314 14775 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8098587abe49f4b71f26b0a9630035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:15 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:42744
I20260926 06:55:15.806331 14909 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8098587abe49f4b71f26b0a9630035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:15 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:49128
I20260926 06:55:15.806536 15017 tablet_replica.cc:330] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58: stopping tablet replica
I20260926 06:55:15.806536 14641 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8098587abe49f4b71f26b0a9630035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:15 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:49442
I20260926 06:55:15.806638 15017 raft_consensus.cc:2229] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:55:15.806722 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 14578
I20260926 06:55:15.806743 15017 raft_consensus.cc:2258] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:55:15.807920 15017 ts_tablet_manager.cc:1905] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:55:15.808146 15016 tablet_replica.cc:330] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b: stopping tablet replica
I20260926 06:55:15.808255 15016 raft_consensus.cc:2229] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:55:15.808353 15016 raft_consensus.cc:2258] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:55:15.808861 15016 ts_tablet_manager.cc:1905] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:55:15.809932 15017 ts_tablet_manager.cc:1918] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:55:15.810021 15017 log.cc:1196] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/4c8098587abe49f4b71f26b0a9630035
I20260926 06:55:15.810273 15017 ts_tablet_manager.cc:1939] T 4c8098587abe49f4b71f26b0a9630035 P 90b4d59ee0484dada801834f1680ad58: Deleting consensus metadata
I20260926 06:55:15.810675 14503 catalog_manager.cc:4841] TS 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645): tablet 4c8098587abe49f4b71f26b0a9630035 (table test-table [id=e72538e43cdc4993bcca5d160596e6c2]) successfully deleted
I20260926 06:55:15.810814 15016 ts_tablet_manager.cc:1918] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:55:15.810885 15016 log.cc:1196] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/4c8098587abe49f4b71f26b0a9630035
I20260926 06:55:15.811146 15016 ts_tablet_manager.cc:1939] T 4c8098587abe49f4b71f26b0a9630035 P 2185dc0f2d344beb82881db37c91d88b: Deleting consensus metadata
I20260926 06:55:15.811542 14502 catalog_manager.cc:4841] TS 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225): tablet 4c8098587abe49f4b71f26b0a9630035 (table test-table [id=e72538e43cdc4993bcca5d160596e6c2]) successfully deleted
W20260926 06:55:15.812852 14502 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:39687 (error 108)
I20260926 06:55:15.813187 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 14712
I20260926 06:55:15.813798 14990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:15.813263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39687 (local address 127.0.0.1:49460)
0926 06:55:15.813724 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:15.813768 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438}
W20260926 06:55:15.813962 14502 catalog_manager.cc:4584] TS aa0e00f504e64287b73bf26a8f445232 (127.0.191.193:39687): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c8098587abe49f4b71f26b0a9630035: Network error: Client connection negotiation failed: client connection to 127.0.191.193:39687: connect: Connection refused (error 111)
I20260926 06:55:15.818840 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 14846
I20260926 06:55:15.823239 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 14484
I20260926 06:55:15.828210 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:39441
--webserver_interface=127.0.191.254
--webserver_port=39995
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:39441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:15.909439 15020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:15.909631 15020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:15.909655 15020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:15.910823 15020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:55:15.910866 15020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:15.910880 15020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:15.910893 15020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:55:15.910941 15020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:55:15.912387 15020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:39441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:39441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=39995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:15.912766 15020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:15.913034 15020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:15.921623 15026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:15.921609 15027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:15.921644 15020 server_base.cc:1034] running on GCE node
W20260926 06:55:15.921609 15029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:15.922057 15020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:15.922423 15020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:15.923568 15020 hybrid_clock.cc:648] HybridClock initialized: now 1790405715923558 us; error 26 us; skew 500 ppm
Sep 26 06:55:15 dist-test-slave-ws74 krb5kdc[14471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:15.929580 15020 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:55:15.929989 15020 webserver.cc:460] Webserver started at http://127.0.191.254:39995/ using document root <none> and password file <none>
I20260926 06:55:15.930188 15020 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:15.930238 15020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:15.931532 15020 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:15.932209 15036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:15.932418 15020 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:15.932507 15020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "26ae31482d134307a6ea0000774f460d"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "de8752365b5f639e41b7348e6d8a88e3"
server_key_iv: "bd6ac8ae324b08edc245d451aa61c2de"
server_key_version: "encryptionkey@0"
I20260926 06:55:15.932888 15020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:15.947075 15020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:15.949299 15020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:15.949504 15020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:15.953104 15020 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:39441
I20260926 06:55:15.953155 15088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:39441 every 8 connection(s)
I20260926 06:55:15.953588 15020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:55:15.955271 15089 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:55:15.955941 15089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Bootstrap starting.
I20260926 06:55:15.962812 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 15020
I20260926 06:55:15.963095 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:39687
--local_ip_for_outbound_sockets=127.0.191.193
--tserver_master_addrs=127.0.191.254:39441
--webserver_port=35095
--webserver_interface=127.0.191.193
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:55:15.964006 15089 log.cc:824] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:15.965988 15089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:55:15.966279 15089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Bootstrap complete.
I20260926 06:55:15.969200 15089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } }
I20260926 06:55:15.969492 15089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26ae31482d134307a6ea0000774f460d, State: Initialized, Role: FOLLOWER
I20260926 06:55:15.969686 15089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } }
I20260926 06:55:15.969758 15089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:55:15.969820 15089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:55:15.969868 15089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 1 FOLLOWER]: Advancing to term 2
I20260926 06:55:15.970830 15089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } }
I20260926 06:55:15.970947 15089 leader_election.cc:302] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26ae31482d134307a6ea0000774f460d; no voters:
I20260926 06:55:15.971135 15089 leader_election.cc:288] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 06:55:15.971172 15093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 2 FOLLOWER]: Leader election won for term 2
I20260926 06:55:15.971290 15093 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 2 LEADER]: Becoming Leader. State: Replica: 26ae31482d134307a6ea0000774f460d, State: Running, Role: LEADER
I20260926 06:55:15.971396 15089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:55:15.971380 15093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } }
I20260926 06:55:15.971648 15093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26ae31482d134307a6ea0000774f460d. Latest consensus state: current_term: 2 leader_uuid: "26ae31482d134307a6ea0000774f460d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } } }
I20260926 06:55:15.971714 15093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:15.971678 15095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "26ae31482d134307a6ea0000774f460d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } } }
I20260926 06:55:15.971905 15097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:55:15.971916 15095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:15.972725 15097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:55:15.972950 15097 catalog_manager.cc:1259] Loaded cluster ID: 76adea3907524256bde19621f97fb390
I20260926 06:55:15.973183 15097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:55:15.973477 15097 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 06:55:15.976292 15097 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 06:55:15.978505 15097 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:55:15.978716 15097 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Loaded TSK: 0
I20260926 06:55:15.978940 15097 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 06:55:16.062875 15091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:16.063107 15091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:16.063145 15091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:16.064299 15091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:16.064365 15091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:16.064419 15091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:55:16.066016 15091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:39687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=35095
--tserver_master_addrs=127.0.191.254:39441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:16.066304 15091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:16.066531 15091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:16.071470 15116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:16.071518 15119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:16.071516 15117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:16.071851 15091 server_base.cc:1034] running on GCE node
I20260926 06:55:16.072067 15091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:16.072379 15091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:16.073544 15091 hybrid_clock.cc:648] HybridClock initialized: now 1790405716073532 us; error 30 us; skew 500 ppm
Sep 26 06:55:16 dist-test-slave-ws74 krb5kdc[14471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405716, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:16.076502 15091 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:55:16.076958 15091 webserver.cc:460] Webserver started at http://127.0.191.193:35095/ using document root <none> and password file <none>
I20260926 06:55:16.077190 15091 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:16.077270 15091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:16.078696 15091 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:16.079442 15126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:16.079612 15091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:16.079710 15091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "aa0e00f504e64287b73bf26a8f445232"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "c08be58152c56ab04f110befc07bd1f5"
server_key_iv: "caf37b5f3bb9a3cdc7f8cec7a0412e91"
server_key_version: "encryptionkey@0"
I20260926 06:55:16.080063 15091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:16.093954 15091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:16.096165 15091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:16.096329 15091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:16.096552 15091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:16.097019 15133 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 06:55:16.098909 15091 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 06:55:16.098968 15091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 06:55:16.099013 15091 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 06:55:16.099603 15091 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 06:55:16.099658 15091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 06:55:16.099730 15133 tablet_bootstrap.cc:492] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Bootstrap starting.
I20260926 06:55:16.106798 15091 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:39687
I20260926 06:55:16.106952 15240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:39687 every 8 connection(s)
I20260926 06:55:16.107226 15091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:55:16.108328 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 15091
I20260926 06:55:16.108623 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:42225
--local_ip_for_outbound_sockets=127.0.191.194
--tserver_master_addrs=127.0.191.254:39441
--webserver_port=44529
--webserver_interface=127.0.191.194
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:55:16.110463 15133 log.cc:824] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:16.112988 15133 tablet_bootstrap.cc:492] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:55:16.113298 15133 tablet_bootstrap.cc:492] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Bootstrap complete.
I20260926 06:55:16.114503 15133 ts_tablet_manager.cc:1397] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260926 06:55:16.116118 15133 raft_consensus.cc:348] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:16.116423 15133 raft_consensus.cc:729] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0e00f504e64287b73bf26a8f445232, State: Initialized, Role: FOLLOWER
I20260926 06:55:16.116575 15133 consensus_queue.cc:260] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
Sep 26 06:55:16 dist-test-slave-ws74 krb5kdc[14471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405716, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:16.116811 15133 ts_tablet_manager.cc:1428] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260926 06:55:16.121677 15243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.107933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:42989)
0926 06:55:16.108274 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:16.108320 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:16.109401 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:16.110008 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:16.110013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:16.110310 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:16.110496 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.110501 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.112223 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.112225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.113611 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.113615 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.114514 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.114515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.114678 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.115322 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:16.115332 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:16.117273 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:16.119650 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:16.119657 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:16.119659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:16.119985 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:16.120290 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:16.120291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:16.120291 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:16.120464 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:16.120906 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:16.120913 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:16.121083 (+ 170us) client_negotiation.cc:769] Sending connection context
0926 06:55:16.121213 (+ 130us) client_negotiation.cc:240] Negotiation successful
0926 06:55:16.121451 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":100,"threads_started":1}
I20260926 06:55:16.121699 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.107996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:42989 (local address 127.0.191.254:39441)
0926 06:55:16.108173 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:16.108183 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:16.108499 (+ 316us) server_negotiation.cc:407] Connection header received
0926 06:55:16.109631 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:16.109639 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:16.109698 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:16.109800 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:16.110686 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.112109 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.113715 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.114394 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.114984 (+ 590us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.117422 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:16.117426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:16.117427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:16.117439 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:16.119555 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:16.120079 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:16.120082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:16.120083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:16.120210 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:16.120545 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:16.120547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:16.120547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:16.120823 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:16.120904 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:16.121279 (+ 375us) server_negotiation.cc:299] Negotiation successful
0926 06:55:16.121454 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260926 06:55:16.122606 15241 heartbeater.cc:347] Connected to a master server at 127.0.191.254:39441
I20260926 06:55:16.122704 15241 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:16.122879 15241 heartbeater.cc:511] Master 127.0.191.254:39441 requested a full tablet report, sending...
I20260926 06:55:16.123648 15053 ts_manager.cc:194] Registered new tserver with Master: aa0e00f504e64287b73bf26a8f445232 (127.0.191.193:39687)
I20260926 06:55:16.124882 15053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:42989
I20260926 06:55:16.125984 15241 heartbeater.cc:503] Master 127.0.191.254:39441 was elected leader, sending a full tablet report...
I20260926 06:55:16.133558 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.125406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49462 (local address 127.0.191.193:39687)
0926 06:55:16.129525 (+ 4119us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:16.129529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:16.129623 (+ 94us) server_negotiation.cc:407] Connection header received
0926 06:55:16.129704 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:16.129706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:16.129722 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:16.129801 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:16.129957 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.131029 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.132432 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.133109 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.133192 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.133245 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:16.133448 (+ 203us) server_negotiation.cc:299] Negotiation successful
0926 06:55:16.133489 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4084,"thread_start_us":68,"threads_started":1}
I20260926 06:55:16.133570 15248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.125297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39687 (local address 127.0.0.1:49462)
0926 06:55:16.129522 (+ 4225us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:16.129555 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:16.129631 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:16.129806 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:16.129826 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:16.129828 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:16.129885 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.129886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.131118 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.131119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.132329 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.132333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.133247 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.133248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.133305 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.133315 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 06:55:16.133464 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 06:55:16.133518 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4195,"thread_start_us":116,"threads_started":1}
I20260926 06:55:16.133966 15175 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8098587abe49f4b71f26b0a9630035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:15 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:49462
I20260926 06:55:16.136729 15250 tablet_replica.cc:330] stopping tablet replica
I20260926 06:55:16.136830 15250 raft_consensus.cc:2229] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:55:16.136896 15250 raft_consensus.cc:2258] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:55:16.137225 15250 ts_tablet_manager.cc:1905] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:55:16.139532 15250 ts_tablet_manager.cc:1918] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:55:16.139607 15250 log.cc:1196] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal/wals/4c8098587abe49f4b71f26b0a9630035
I20260926 06:55:16.139820 15250 ts_tablet_manager.cc:1939] T 4c8098587abe49f4b71f26b0a9630035 P aa0e00f504e64287b73bf26a8f445232: Deleting consensus metadata
I20260926 06:55:16.140467 15039 catalog_manager.cc:4841] TS aa0e00f504e64287b73bf26a8f445232 (127.0.191.193:39687): tablet 4c8098587abe49f4b71f26b0a9630035 (table test-table [id=e72538e43cdc4993bcca5d160596e6c2]) successfully deleted
W20260926 06:55:16.197669 15245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:16.197902 15245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:16.197942 15245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:16.199096 15245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:16.199167 15245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:16.199211 15245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:55:16.200675 15245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:42225
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=44529
--tserver_master_addrs=127.0.191.254:39441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:16.200949 15245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:16.201184 15245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:16.206302 15258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:16.206382 15256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:16.206499 15245 server_base.cc:1034] running on GCE node
W20260926 06:55:16.206357 15255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:16.206755 15245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:16.206966 15245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:16.208128 15245 hybrid_clock.cc:648] HybridClock initialized: now 1790405716208120 us; error 34 us; skew 500 ppm
Sep 26 06:55:16 dist-test-slave-ws74 krb5kdc[14471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405716, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:16.210737 15245 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:55:16.211181 15245 webserver.cc:460] Webserver started at http://127.0.191.194:44529/ using document root <none> and password file <none>
I20260926 06:55:16.211380 15245 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:16.211432 15245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:16.212687 15245 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:16.213228 15265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:16.213488 15245 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:16.213582 15245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "2185dc0f2d344beb82881db37c91d88b"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "d174c0acb79d6c60a43d44e146690883"
server_key_iv: "5a818f67a218b2087fa09443aa5bd8bc"
server_key_version: "encryptionkey@0"
I20260926 06:55:16.213927 15245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:16.220563 15245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:16.222419 15245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:16.222575 15245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:16.222793 15245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:16.223095 15245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:16.223150 15245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:16.223205 15245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:16.223234 15245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:16.228854 15245 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:42225
I20260926 06:55:16.228894 15378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:42225 every 8 connection(s)
I20260926 06:55:16.229256 15245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:55:16 dist-test-slave-ws74 krb5kdc[14471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405716, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:16.235455 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 15245
I20260926 06:55:16.235792 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:35645
--local_ip_for_outbound_sockets=127.0.191.195
--tserver_master_addrs=127.0.191.254:39441
--webserver_port=34101
--webserver_interface=127.0.191.195
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:55:16.239056 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.230022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:57989 (local address 127.0.191.254:39441)
0926 06:55:16.230083 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:16.230085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:16.230297 (+ 212us) server_negotiation.cc:407] Connection header received
0926 06:55:16.230825 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:16.230826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:16.230841 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:16.230894 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:16.231812 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.232317 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.233408 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.233615 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.233730 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.235922 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:16.235924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:16.235924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:16.235931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:16.237686 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:16.238073 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:16.238074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:16.238075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:16.238138 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:16.238428 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:16.238428 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:16.238429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:16.238531 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:16.238584 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:16.238923 (+ 339us) server_negotiation.cc:299] Negotiation successful
0926 06:55:16.238992 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:55:16.239355 15381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.229995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:57989)
0926 06:55:16.230171 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:16.230212 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:16.230719 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:16.230970 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:16.230974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:16.231527 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:16.231708 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.231713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.232450 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.232451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.233282 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.233284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.233733 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.233734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.233865 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.234383 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:16.234392 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:16.235797 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:16.237766 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:16.237780 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:16.237782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:16.237993 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:16.238216 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:16.238217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:16.238217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:16.238356 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:16.238598 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:16.238604 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:16.238787 (+ 183us) client_negotiation.cc:769] Sending connection context
0926 06:55:16.238903 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 06:55:16.239076 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260926 06:55:16.239892 15379 heartbeater.cc:347] Connected to a master server at 127.0.191.254:39441
I20260926 06:55:16.240018 15379 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:16.240213 15379 heartbeater.cc:511] Master 127.0.191.254:39441 requested a full tablet report, sending...
I20260926 06:55:16.240592 15053 ts_manager.cc:194] Registered new tserver with Master: 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225)
I20260926 06:55:16.240971 15053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:57989
W20260926 06:55:16.319336 15382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:16.319535 15382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:16.319557 15382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:16.320665 15382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:16.320715 15382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:16.320758 15382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:55:16.322218 15382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:35645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=34101
--tserver_master_addrs=127.0.191.254:39441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:16.322409 15382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:16.322585 15382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:16.327368 15388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:16.327471 15387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:16.327378 15390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:16.327664 15382 server_base.cc:1034] running on GCE node
I20260926 06:55:16.327867 15382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:16.328053 15382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:16.329192 15382 hybrid_clock.cc:648] HybridClock initialized: now 1790405716329173 us; error 39 us; skew 500 ppm
Sep 26 06:55:16 dist-test-slave-ws74 krb5kdc[14471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405716, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:16.331792 15382 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:55:16.332207 15382 webserver.cc:460] Webserver started at http://127.0.191.195:34101/ using document root <none> and password file <none>
I20260926 06:55:16.332432 15382 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:16.332486 15382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:16.333915 15382 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:16.334491 15397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:16.334657 15382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:16.334756 15382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "90b4d59ee0484dada801834f1680ad58"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "aec471c60d83b2e7b6cafaa2efcdfcc9"
server_key_iv: "942b94a21454449542539410417e425b"
server_key_version: "encryptionkey@0"
I20260926 06:55:16.335127 15382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:16.346985 15382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:16.348878 15382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:16.349043 15382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:16.349277 15382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:16.349599 15382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:16.349653 15382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:16.349694 15382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:16.349722 15382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:16.355525 15382 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:35645
I20260926 06:55:16.355597 15510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:35645 every 8 connection(s)
I20260926 06:55:16.355947 15382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:55:16.361403 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 15382
Sep 26 06:55:16 dist-test-slave-ws74 krb5kdc[14471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405716, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:16.365406 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.356668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:41621 (local address 127.0.191.254:39441)
0926 06:55:16.356723 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:16.356726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:16.356941 (+ 215us) server_negotiation.cc:407] Connection header received
0926 06:55:16.357443 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:16.357444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:16.357455 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:16.357523 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:16.358399 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.358871 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.359828 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.360010 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.360059 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.362340 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:16.362341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:16.362341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:16.362349 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:16.363906 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:16.364275 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:16.364276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:16.364277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:16.364339 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:16.364606 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:16.364607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:16.364607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:16.364750 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:16.364826 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:16.365156 (+ 330us) server_negotiation.cc:299] Negotiation successful
0926 06:55:16.365236 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:55:16.365461 15513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.356663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:41621)
0926 06:55:16.356816 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:16.356850 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:16.357335 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:16.357578 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:16.357581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:16.358127 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:16.358321 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.358325 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.358955 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.358956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.359748 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.359750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.360079 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.360080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.360175 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.360665 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:16.360673 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:16.362255 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:16.363983 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:16.363988 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:16.363990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:16.364194 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:16.364411 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:16.364412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:16.364413 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:16.364540 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:16.364821 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:16.364826 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:16.364990 (+ 164us) client_negotiation.cc:769] Sending connection context
0926 06:55:16.365106 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 06:55:16.365245 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":92,"threads_started":1}
I20260926 06:55:16.365878 15511 heartbeater.cc:347] Connected to a master server at 127.0.191.254:39441
I20260926 06:55:16.365967 15511 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:16.366153 15511 heartbeater.cc:511] Master 127.0.191.254:39441 requested a full tablet report, sending...
I20260926 06:55:16.366541 15053 ts_manager.cc:194] Registered new tserver with Master: 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645)
I20260926 06:55:16.366976 15053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:41621
I20260926 06:55:16.368631 15515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.361834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58046 (local address 127.0.191.254:39441)
0926 06:55:16.361948 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:16.361950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:16.362029 (+ 79us) server_negotiation.cc:407] Connection header received
0926 06:55:16.362059 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:16.362060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:16.362072 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:16.362144 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:16.362628 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.363308 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.364308 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.364585 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.364646 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.366076 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:16.366077 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:16.366077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:16.366084 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:16.367754 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:16.368068 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:16.368068 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:16.368069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:16.368118 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:16.368243 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:16.368243 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:16.368243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:16.368356 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:16.368472 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:16.368517 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 06:55:16.368574 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":54,"threads_started":1}
I20260926 06:55:16.369248 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:55:16.371703 15515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.369606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58060 (local address 127.0.191.254:39441)
0926 06:55:16.369640 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:16.369641 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:16.369717 (+ 76us) server_negotiation.cc:407] Connection header received
0926 06:55:16.369772 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:16.369772 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:16.369796 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:16.369855 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:55:16.369929 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.370354 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.371038 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.371216 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.371268 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.371461 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:55:16.371542 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:55:16.371601 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:16.371623 (+ 22us) server_negotiation.cc:299] Negotiation successful
0926 06:55:16.371640 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:55:16.371929 15053 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:55:16.373649 15053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:55:16.375829 15175 tablet_service.cc:1467] Processing CreateTablet for tablet 6530499a14844ce68f97c23d46d425e3 (DEFAULT_TABLE table=test-table [id=a5dd519c0a0e4fc698cf96eab2f6e319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:16.376045 15175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6530499a14844ce68f97c23d46d425e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:16.378463 15133 tablet_bootstrap.cc:492] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232: Bootstrap starting.
I20260926 06:55:16.378783 15133 tablet_bootstrap.cc:654] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:16.379339 15133 tablet_bootstrap.cc:492] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232: No bootstrap required, opened a new log
I20260926 06:55:16.379390 15133 ts_tablet_manager.cc:1397] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 06:55:16.379503 15133 raft_consensus.cc:348] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:16.379577 15133 raft_consensus.cc:374] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:16.379562 15248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.375494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:42225 (local address 127.0.0.1:42754)
0926 06:55:16.375527 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:16.375537 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:16.375574 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:16.375978 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:16.375979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:16.375981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:16.376017 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.376018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.377247 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.377247 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.378336 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.378338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.379326 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.379327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.379394 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.379397 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:55:16.379446 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 06:55:16.379492 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":2}
I20260926 06:55:16.379629 15133 raft_consensus.cc:729] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0e00f504e64287b73bf26a8f445232, State: Initialized, Role: FOLLOWER
I20260926 06:55:16.379675 15133 consensus_queue.cc:260] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:16.379736 15519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.375710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49130 (local address 127.0.191.195:35645)
0926 06:55:16.376072 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:16.376075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:16.376082 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:55:16.376129 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:16.376131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:16.376145 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:16.376211 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:16.376421 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.377072 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.378447 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.379160 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.379268 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.379316 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:16.379610 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 06:55:16.379657 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":108,"threads_started":1}
I20260926 06:55:16.379761 15133 ts_tablet_manager.cc:1428] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:16.379791 15517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.375535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:35645 (local address 127.0.0.1:49130)
0926 06:55:16.375732 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:16.375741 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:16.375788 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:16.376224 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:16.376225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:16.376227 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:16.376329 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.376331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.377169 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.377170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.378371 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.378373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.379356 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.379357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.379426 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.379431 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:55:16.379492 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 06:55:16.379547 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260926 06:55:16.380168 15518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.375584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42754 (local address 127.0.191.194:42225)
0926 06:55:16.375792 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:16.375815 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:16.375827 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:55:16.375866 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:16.375868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:16.375883 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:16.375970 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:16.376110 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.377148 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.378447 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.379160 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.379236 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.379293 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:16.380039 (+ 746us) server_negotiation.cc:299] Negotiation successful
0926 06:55:16.380086 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":97,"threads_started":1}
I20260926 06:55:16.380446 15445 tablet_service.cc:1467] Processing CreateTablet for tablet 6530499a14844ce68f97c23d46d425e3 (DEFAULT_TABLE table=test-table [id=a5dd519c0a0e4fc698cf96eab2f6e319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:16.380745 15445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6530499a14844ce68f97c23d46d425e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:16.380812 15313 tablet_service.cc:1467] Processing CreateTablet for tablet 6530499a14844ce68f97c23d46d425e3 (DEFAULT_TABLE table=test-table [id=a5dd519c0a0e4fc698cf96eab2f6e319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:16.381109 15313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6530499a14844ce68f97c23d46d425e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:16.383509 15521 tablet_bootstrap.cc:492] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58: Bootstrap starting.
I20260926 06:55:16.384145 15521 tablet_bootstrap.cc:654] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:16.384541 15521 log.cc:824] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:16.384797 15522 tablet_bootstrap.cc:492] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b: Bootstrap starting.
I20260926 06:55:16.385191 15521 tablet_bootstrap.cc:492] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58: No bootstrap required, opened a new log
I20260926 06:55:16.385308 15521 ts_tablet_manager.cc:1397] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:55:16.386695 15521 raft_consensus.cc:348] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:16.386859 15521 raft_consensus.cc:374] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:16.386885 15521 raft_consensus.cc:729] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90b4d59ee0484dada801834f1680ad58, State: Initialized, Role: FOLLOWER
I20260926 06:55:16.387018 15521 consensus_queue.cc:260] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:16.387300 15521 ts_tablet_manager.cc:1428] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:55:16.387305 15511 heartbeater.cc:503] Master 127.0.191.254:39441 was elected leader, sending a full tablet report...
I20260926 06:55:16.387866 15522 tablet_bootstrap.cc:654] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:16.388404 15522 log.cc:824] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:16.389559 15522 tablet_bootstrap.cc:492] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b: No bootstrap required, opened a new log
I20260926 06:55:16.389616 15522 ts_tablet_manager.cc:1397] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260926 06:55:16.391144 15522 raft_consensus.cc:348] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:16.391283 15522 raft_consensus.cc:374] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:16.391306 15522 raft_consensus.cc:729] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2185dc0f2d344beb82881db37c91d88b, State: Initialized, Role: FOLLOWER
I20260926 06:55:16.391439 15522 consensus_queue.cc:260] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:16.391700 15522 ts_tablet_manager.cc:1428] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:55:16.391746 15379 heartbeater.cc:503] Master 127.0.191.254:39441 was elected leader, sending a full tablet report...
W20260926 06:55:16.479833 15380 tablet.cc:2369] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:55:16.606498 15512 tablet.cc:2369] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:55:16.608121 15242 tablet.cc:2369] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:55:16.652961 15524 raft_consensus.cc:482] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:55:16.653146 15524 raft_consensus.cc:504] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:16.653594 15524 leader_election.cc:288] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225), aa0e00f504e64287b73bf26a8f445232 (127.0.191.193:39687)
I20260926 06:55:16.657624 15518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.653862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:58565 (local address 127.0.191.194:42225)
0926 06:55:16.653926 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:16.653928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:16.653942 (+ 14us) server_negotiation.cc:407] Connection header received
0926 06:55:16.653976 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:16.653977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:16.653988 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:16.654067 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:16.654315 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.655102 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.656624 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.657190 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.657270 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.657278 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:16.657515 (+ 237us) server_negotiation.cc:299] Negotiation successful
0926 06:55:16.657554 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 06:55:16.657624 15528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.654009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:52135 (local address 127.0.191.193:39687)
0926 06:55:16.654135 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:16.654137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:16.654146 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:55:16.654186 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:16.654186 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:16.654195 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:16.654242 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:16.654419 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.655133 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.656624 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.657190 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.657270 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.657278 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:16.657514 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 06:55:16.657555 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260926 06:55:16.657639 15513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.653780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:42225 (local address 127.0.191.195:58565)
0926 06:55:16.653841 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:16.653854 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:16.653902 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:16.654099 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:16.654100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:16.654101 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:16.654174 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.654176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.655225 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.655226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.656535 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.656538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.657313 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.657314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.657429 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.657432 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:55:16.657499 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 06:55:16.657545 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":63}
I20260926 06:55:16.657645 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.653917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39687 (local address 127.0.191.195:52135)
0926 06:55:16.654081 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:16.654090 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:16.654133 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:16.654265 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:16.654266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:16.654268 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:16.654333 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.654334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.655197 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.655198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.656535 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:16.656538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.657312 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.657314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:16.657428 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.657432 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:55:16.657499 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 06:55:16.657544 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":77,"thread_start_us":118,"threads_started":1}
I20260926 06:55:16.657990 15333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6530499a14844ce68f97c23d46d425e3" candidate_uuid: "90b4d59ee0484dada801834f1680ad58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2185dc0f2d344beb82881db37c91d88b" is_pre_election: true
I20260926 06:55:16.658126 15333 raft_consensus.cc:2454] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90b4d59ee0484dada801834f1680ad58 in term 0.
I20260926 06:55:16.658367 15400 leader_election.cc:302] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2185dc0f2d344beb82881db37c91d88b, 90b4d59ee0484dada801834f1680ad58; no voters:
I20260926 06:55:16.658475 15195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6530499a14844ce68f97c23d46d425e3" candidate_uuid: "90b4d59ee0484dada801834f1680ad58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0e00f504e64287b73bf26a8f445232" is_pre_election: true
I20260926 06:55:16.658521 15524 raft_consensus.cc:2781] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:55:16.658591 15524 raft_consensus.cc:482] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:55:16.658614 15524 raft_consensus.cc:3037] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:16.658641 15195 raft_consensus.cc:2454] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90b4d59ee0484dada801834f1680ad58 in term 0.
I20260926 06:55:16.659355 15524 raft_consensus.cc:504] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:16.659466 15524 leader_election.cc:288] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [CANDIDATE]: Term 1 election: Requested vote from peers 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225), aa0e00f504e64287b73bf26a8f445232 (127.0.191.193:39687)
I20260926 06:55:16.659646 15195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6530499a14844ce68f97c23d46d425e3" candidate_uuid: "90b4d59ee0484dada801834f1680ad58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0e00f504e64287b73bf26a8f445232"
I20260926 06:55:16.659669 15333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6530499a14844ce68f97c23d46d425e3" candidate_uuid: "90b4d59ee0484dada801834f1680ad58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2185dc0f2d344beb82881db37c91d88b"
I20260926 06:55:16.659729 15195 raft_consensus.cc:3037] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:16.659756 15333 raft_consensus.cc:3037] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:16.660457 15195 raft_consensus.cc:2454] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90b4d59ee0484dada801834f1680ad58 in term 1.
I20260926 06:55:16.660611 15400 leader_election.cc:302] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90b4d59ee0484dada801834f1680ad58, aa0e00f504e64287b73bf26a8f445232; no voters:
I20260926 06:55:16.660633 15333 raft_consensus.cc:2454] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90b4d59ee0484dada801834f1680ad58 in term 1.
I20260926 06:55:16.660691 15524 raft_consensus.cc:2781] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:55:16.660836 15524 raft_consensus.cc:686] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [term 1 LEADER]: Becoming Leader. State: Replica: 90b4d59ee0484dada801834f1680ad58, State: Running, Role: LEADER
I20260926 06:55:16.660926 15524 consensus_queue.cc:237] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:16.661690 15053 catalog_manager.cc:5495] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90b4d59ee0484dada801834f1680ad58 (127.0.191.195). New cstate: current_term: 1 leader_uuid: "90b4d59ee0484dada801834f1680ad58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } health_report { overall_health: UNKNOWN } } }
I20260926 06:55:16.734354 15519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.732121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49138 (local address 127.0.191.195:35645)
0926 06:55:16.732182 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:16.732184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:16.732199 (+ 15us) server_negotiation.cc:407] Connection header received
0926 06:55:16.732234 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:16.732235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:16.732252 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:16.732321 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:55:16.732422 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.732882 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.733654 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:16.733850 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:16.733903 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:16.734037 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:55:16.734131 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:55:16.734185 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:16.734220 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 06:55:16.734235 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 06:55:16.736212 15195 raft_consensus.cc:1261] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Refusing update from remote peer 90b4d59ee0484dada801834f1680ad58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:55:16.736212 15333 raft_consensus.cc:1261] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b [term 1 FOLLOWER]: Refusing update from remote peer 90b4d59ee0484dada801834f1680ad58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:55:16.736567 15524 consensus_queue.cc:1035] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:55:16.736739 15529 consensus_queue.cc:1035] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:55:16.740866 15052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:55:16.740995 15052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:55:16.741747 15052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Sending DeleteTablet for 3 replicas of tablet 6530499a14844ce68f97c23d46d425e3
I20260926 06:55:16.742033 15313 tablet_service.cc:1514] Processing DeleteTablet for tablet 6530499a14844ce68f97c23d46d425e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:16 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:42754
I20260926 06:55:16.742069 15175 tablet_service.cc:1514] Processing DeleteTablet for tablet 6530499a14844ce68f97c23d46d425e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:16 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:49462
I20260926 06:55:16.742089 15445 tablet_service.cc:1514] Processing DeleteTablet for tablet 6530499a14844ce68f97c23d46d425e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:16 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:49130
I20260926 06:55:16.742352 15542 tablet_replica.cc:330] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b: stopping tablet replica
I20260926 06:55:16.742354 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 15091
I20260926 06:55:16.743052 15542 raft_consensus.cc:2229] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:55:16.743185 15542 raft_consensus.cc:2258] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:55:16.743628 15542 ts_tablet_manager.cc:1905] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:55:16.745435 15542 ts_tablet_manager.cc:1918] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:55:16.745510 15542 log.cc:1196] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/6530499a14844ce68f97c23d46d425e3
I20260926 06:55:16.745522 15544 tablet_replica.cc:330] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58: stopping tablet replica
I20260926 06:55:16.745626 15544 raft_consensus.cc:2229] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:55:16.745749 15544 raft_consensus.cc:2258] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:55:16.745764 15542 ts_tablet_manager.cc:1939] T 6530499a14844ce68f97c23d46d425e3 P 2185dc0f2d344beb82881db37c91d88b: Deleting consensus metadata
I20260926 06:55:16.746152 15039 catalog_manager.cc:4841] TS 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225): tablet 6530499a14844ce68f97c23d46d425e3 (table test-table [id=a5dd519c0a0e4fc698cf96eab2f6e319]) successfully deleted
I20260926 06:55:16.746307 15544 ts_tablet_manager.cc:1905] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:55:16.748418 15544 ts_tablet_manager.cc:1918] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:55:16.748498 15544 log.cc:1196] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/6530499a14844ce68f97c23d46d425e3
I20260926 06:55:16.748795 15544 ts_tablet_manager.cc:1939] T 6530499a14844ce68f97c23d46d425e3 P 90b4d59ee0484dada801834f1680ad58: Deleting consensus metadata
I20260926 06:55:16.749193 15040 catalog_manager.cc:4841] TS 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645): tablet 6530499a14844ce68f97c23d46d425e3 (table test-table [id=a5dd519c0a0e4fc698cf96eab2f6e319]) successfully deleted
W20260926 06:55:16.750068 15039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:39687 (error 108)
I20260926 06:55:16.750517 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 15245
I20260926 06:55:16.750564 15517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:16.750434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39687 (local address 127.0.0.1:49474)
0926 06:55:16.750484 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:16.750529 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260926 06:55:16.752277 15039 catalog_manager.cc:4584] TS aa0e00f504e64287b73bf26a8f445232 (127.0.191.193:39687): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6530499a14844ce68f97c23d46d425e3: Network error: Client connection negotiation failed: client connection to 127.0.191.193:39687: connect: Connection refused (error 111)
I20260926 06:55:16.756232 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 15382
I20260926 06:55:16.761830 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 15020
I20260926 06:55:16.767180 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:39441
--webserver_interface=127.0.191.254
--webserver_port=39995
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:39441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:16.850268 15546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:16.850515 15546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:16.850554 15546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:16.851693 15546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:55:16.851760 15546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:16.851784 15546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:16.851804 15546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:55:16.851835 15546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:55:16.853276 15546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:39441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:39441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=39995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:16.853570 15546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:16.853787 15546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:16.858664 15552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:16.858785 15546 server_base.cc:1034] running on GCE node
W20260926 06:55:16.858690 15554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:16.858904 15551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:16.859129 15546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:16.859414 15546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:16.860579 15546 hybrid_clock.cc:648] HybridClock initialized: now 1790405716860570 us; error 43 us; skew 500 ppm
Sep 26 06:55:16 dist-test-slave-ws74 krb5kdc[14471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405716, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:16.863041 15546 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:55:16.863505 15546 webserver.cc:460] Webserver started at http://127.0.191.254:39995/ using document root <none> and password file <none>
I20260926 06:55:16.863739 15546 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:16.863785 15546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:16.865087 15546 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:16.865832 15561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:16.866014 15546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:16.866104 15546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "26ae31482d134307a6ea0000774f460d"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "de8752365b5f639e41b7348e6d8a88e3"
server_key_iv: "bd6ac8ae324b08edc245d451aa61c2de"
server_key_version: "encryptionkey@0"
I20260926 06:55:16.866443 15546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:16.891525 15546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:16.893890 15546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:16.894038 15546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:16.898236 15546 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:39441
I20260926 06:55:16.898277 15613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:39441 every 8 connection(s)
I20260926 06:55:16.898689 15546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:55:16.900180 15614 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:55:16.900774 15614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Bootstrap starting.
I20260926 06:55:16.902050 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 15546
I20260926 06:55:16.902302 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:39687
--local_ip_for_outbound_sockets=127.0.191.193
--tserver_master_addrs=127.0.191.254:39441
--webserver_port=35095
--webserver_interface=127.0.191.193
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:55:16.909122 15614 log.cc:824] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:16.911300 15614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:55:16.911608 15614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d: Bootstrap complete.
I20260926 06:55:16.913969 15614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } }
I20260926 06:55:16.914196 15614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26ae31482d134307a6ea0000774f460d, State: Initialized, Role: FOLLOWER
I20260926 06:55:16.914383 15614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } }
I20260926 06:55:16.914438 15614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:55:16.914497 15614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:55:16.914543 15614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 2 FOLLOWER]: Advancing to term 3
I20260926 06:55:16.915473 15614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } }
I20260926 06:55:16.915591 15614 leader_election.cc:302] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26ae31482d134307a6ea0000774f460d; no voters:
I20260926 06:55:16.915798 15614 leader_election.cc:288] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 06:55:16.915838 15618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 3 FOLLOWER]: Leader election won for term 3
I20260926 06:55:16.915949 15618 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [term 3 LEADER]: Becoming Leader. State: Replica: 26ae31482d134307a6ea0000774f460d, State: Running, Role: LEADER
I20260926 06:55:16.916064 15614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:55:16.916114 15618 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } }
I20260926 06:55:16.916385 15618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26ae31482d134307a6ea0000774f460d. Latest consensus state: current_term: 3 leader_uuid: "26ae31482d134307a6ea0000774f460d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } } }
I20260926 06:55:16.916464 15618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:16.916599 15619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "26ae31482d134307a6ea0000774f460d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae31482d134307a6ea0000774f460d" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39441 } } }
I20260926 06:55:16.916672 15619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26ae31482d134307a6ea0000774f460d [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:16.916972 15624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:55:16.917762 15624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:55:16.918098 15624 catalog_manager.cc:1259] Loaded cluster ID: 76adea3907524256bde19621f97fb390
I20260926 06:55:16.918138 15624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 06:55:16.918444 15624 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f436c66c9ad google::LogMessage::Fail() at ??:0
@ 0x7f436c66f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f436c66c4de google::LogMessage::Flush() at ??:0
@ 0x7f436c66da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f43745920f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f437459b8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f436e53a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f436e52f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f436f3936db start_thread at ??:0
@ 0x7f436b16a61f clone at ??:0
W20260926 06:55:16.996369 15616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:16.996557 15616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:16.996580 15616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:16.997781 15616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:16.997821 15616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:16.997848 15616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:55:16.999275 15616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:39687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=35095
--tserver_master_addrs=127.0.191.254:39441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:16.999501 15616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:16.999711 15616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:17.005048 15651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:17.005183 15616 server_base.cc:1034] running on GCE node
W20260926 06:55:17.005059 15653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:17.005071 15650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:17.005599 15616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:17.005872 15616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:17.007028 15616 hybrid_clock.cc:648] HybridClock initialized: now 1790405717007005 us; error 41 us; skew 500 ppm
Sep 26 06:55:17 dist-test-slave-ws74 krb5kdc[14471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405717, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:17.009549 15616 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:55:17.010012 15616 webserver.cc:460] Webserver started at http://127.0.191.193:35095/ using document root <none> and password file <none>
I20260926 06:55:17.010249 15616 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:17.010295 15616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:17.011592 15616 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:17.012290 15660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:17.012472 15616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:17.012575 15616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "aa0e00f504e64287b73bf26a8f445232"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "c08be58152c56ab04f110befc07bd1f5"
server_key_iv: "caf37b5f3bb9a3cdc7f8cec7a0412e91"
server_key_version: "encryptionkey@0"
I20260926 06:55:17.012918 15616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:17.024446 15616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:17.026731 15616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:17.026875 15616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:17.027115 15616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:17.027575 15667 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 06:55:17.029493 15616 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 06:55:17.029552 15616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 06:55:17.029592 15616 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 06:55:17.030186 15616 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 06:55:17.030241 15616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 06:55:17.030309 15667 tablet_bootstrap.cc:492] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232: Bootstrap starting.
I20260926 06:55:17.037433 15616 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:39687
I20260926 06:55:17.037449 15774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:39687 every 8 connection(s)
I20260926 06:55:17.037837 15616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:55:17.041551 15667 log.cc:824] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:17.044337 15667 tablet_bootstrap.cc:492] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:55:17.044678 15667 tablet_bootstrap.cc:492] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232: Bootstrap complete.
I20260926 06:55:17.045964 15667 ts_tablet_manager.cc:1397] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.010s
I20260926 06:55:17.047538 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 15616
I20260926 06:55:17.047372 15667 raft_consensus.cc:348] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:17.047714 15667 raft_consensus.cc:729] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0e00f504e64287b73bf26a8f445232, State: Initialized, Role: FOLLOWER
I20260926 06:55:17.047871 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:42225
--local_ip_for_outbound_sockets=127.0.191.194
--tserver_master_addrs=127.0.191.254:39441
--webserver_port=44529
--webserver_interface=127.0.191.194
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:55:17.047888 15667 consensus_queue.cc:260] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:17.048169 15667 ts_tablet_manager.cc:1428] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260926 06:55:17.119683 15777 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:55:17.038589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:49391)
0926 06:55:17.038772 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:17.038806 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:17.039229 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:17.119472 (+ 80243us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
W20260926 06:55:17.119901 15775 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:39441 (0 consecutive failures): Network error: Failed to ping master at 127.0.191.254:39441: Client connection negotiation failed: client connection to 127.0.191.254:39441: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 06:55:17.120229 15777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:17.120108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:40025)
0926 06:55:17.120164 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:17.120201 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 06:55:17.120596 15777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:17.120510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:44937)
0926 06:55:17.120557 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:17.120581 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260926 06:55:17.120738 15775 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 06:55:17.132256 15780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:17.132500 15780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:17.132526 15780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:17.133788 15780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:17.133827 15780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:17.133873 15780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:55:17.135350 15780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:42225
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=44529
--tserver_master_addrs=127.0.191.254:39441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:17.135578 15780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:17.135797 15780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:17.140633 15785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:17.140651 15786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:17.140861 15780 server_base.cc:1034] running on GCE node
W20260926 06:55:17.140671 15788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:17.141121 15780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:17.141348 15780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:17.142552 15780 hybrid_clock.cc:648] HybridClock initialized: now 1790405717142536 us; error 26 us; skew 500 ppm
Sep 26 06:55:17 dist-test-slave-ws74 krb5kdc[14471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405717, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:17.145049 15780 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:55:17.145550 15780 webserver.cc:460] Webserver started at http://127.0.191.194:44529/ using document root <none> and password file <none>
I20260926 06:55:17.145728 15780 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:17.145768 15780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:17.146970 15780 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:17.147500 15795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:17.147680 15780 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:55:17.147748 15780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "2185dc0f2d344beb82881db37c91d88b"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "d174c0acb79d6c60a43d44e146690883"
server_key_iv: "5a818f67a218b2087fa09443aa5bd8bc"
server_key_version: "encryptionkey@0"
I20260926 06:55:17.148084 15780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:17.169766 15780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:17.172060 15780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:17.172222 15780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:17.172449 15780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:17.172760 15780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:17.172840 15780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:17.172874 15780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:17.172912 15780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:17.178730 15780 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:42225
I20260926 06:55:17.178777 15908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:42225 every 8 connection(s)
I20260926 06:55:17.179101 15780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:55:17.180078 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:17.179734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:38359)
0926 06:55:17.179875 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:17.179941 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
W20260926 06:55:17.180490 15909 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:39441 (0 consecutive failures): Network error: Failed to ping master at 127.0.191.254:39441: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
I20260926 06:55:17.180774 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:17.180687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:42835)
0926 06:55:17.180731 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:17.180757 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 06:55:17.181066 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:17.180997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:60869)
0926 06:55:17.181032 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:17.181052 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260926 06:55:17.181183 15909 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 06:55:17.183072 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 15780
I20260926 06:55:17.183341 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:35645
--local_ip_for_outbound_sockets=127.0.191.195
--tserver_master_addrs=127.0.191.254:39441
--webserver_port=34101
--webserver_interface=127.0.191.195
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:17.263023 15912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:17.263285 15912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:17.263331 15912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:17.264467 15912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:17.264544 15912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:17.264587 15912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:55:17.266081 15912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:35645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=34101
--tserver_master_addrs=127.0.191.254:39441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:17.266338 15912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:17.266561 15912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:17.271539 15918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:17.271548 15920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:17.271692 15912 server_base.cc:1034] running on GCE node
W20260926 06:55:17.271593 15917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:17.272009 15912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:17.272204 15912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:17.273366 15912 hybrid_clock.cc:648] HybridClock initialized: now 1790405717273349 us; error 34 us; skew 500 ppm
Sep 26 06:55:17 dist-test-slave-ws74 krb5kdc[14471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405717, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:17.275759 15912 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:55:17.276129 15912 webserver.cc:460] Webserver started at http://127.0.191.195:34101/ using document root <none> and password file <none>
I20260926 06:55:17.276371 15912 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:17.276419 15912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:17.277647 15912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:17.278180 15927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:17.278394 15912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:17.278473 15912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "90b4d59ee0484dada801834f1680ad58"
format_stamp: "Formatted at 2026-09-26 06:55:15 on dist-test-slave-ws74"
server_key: "aec471c60d83b2e7b6cafaa2efcdfcc9"
server_key_iv: "942b94a21454449542539410417e425b"
server_key_version: "encryptionkey@0"
I20260926 06:55:17.278745 15912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:17.289798 15912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:17.291534 15912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:17.291638 15912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:17.291797 15912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:17.292035 15912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:17.292065 15912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:17.292111 15912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:17.292138 15912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:17.297833 15912 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:35645
I20260926 06:55:17.297914 16040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:35645 every 8 connection(s)
I20260926 06:55:17.298251 15912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:55:17.299226 16043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:17.298877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:47771)
0926 06:55:17.299027 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:17.299092 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
W20260926 06:55:17.299634 16041 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:39441 (0 consecutive failures): Network error: Failed to ping master at 127.0.191.254:39441: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
I20260926 06:55:17.299887 16043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:17.299800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:41327)
0926 06:55:17.299844 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:17.299870 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 06:55:17.300146 16043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:17.300078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:60451)
0926 06:55:17.300114 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:17.300134 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260926 06:55:17.300262 16041 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 06:55:17.308156 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 15912
I20260926 06:55:18.121306 16045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:18.121068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:35219)
0926 06:55:18.121201 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:18.121237 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260926 06:55:18.181730 16046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:18.181541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:49417)
0926 06:55:18.181652 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:18.181684 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260926 06:55:18.300853 16047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:18.300580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:59065)
0926 06:55:18.300768 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:18.300805 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":51,"threads_started":1}
I20260926 06:55:18.322805 16048 raft_consensus.cc:482] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:55:18.322924 16048 raft_consensus.cc:504] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:18.323299 16048 leader_election.cc:288] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225), 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645)
Sep 26 06:55:18 dist-test-slave-ws74 krb5kdc[14471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405717, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.195@KRBTEST.COM
Sep 26 06:55:18 dist-test-slave-ws74 krb5kdc[14471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405717, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.194@KRBTEST.COM
I20260926 06:55:18.333351 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:18.323592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:53161 (local address 127.0.191.195:35645)
0926 06:55:18.323707 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:18.323719 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:18.323758 (+ 39us) server_negotiation.cc:407] Connection header received
0926 06:55:18.324178 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:18.324181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:18.324220 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:18.324375 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:18.325078 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:18.326082 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:18.327037 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:18.327437 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:18.327562 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:18.329745 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:18.329753 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:18.329754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:18.329765 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:18.331632 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:18.332172 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:18.332175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:18.332175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:18.332269 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:18.332520 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:18.332521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:18.332521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:18.332739 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:18.332832 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:18.333108 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 06:55:18.333232 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":58,"threads_started":1}
I20260926 06:55:18.333354 16045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:18.323450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:35645 (local address 127.0.191.193:53161)
0926 06:55:18.323511 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:18.323524 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:18.323591 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:18.324411 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:18.324414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:18.324766 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:18.324920 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:18.324925 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:18.326182 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:18.326183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:18.326945 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:18.326947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:18.327526 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:18.327527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:18.327636 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:18.328203 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:18.328210 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:18.329637 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:18.331752 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:18.331765 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:18.331767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:18.332075 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:18.332345 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:18.332346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:18.332346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:18.332455 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:18.332813 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:18.332819 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:18.332948 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 06:55:18.333049 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 06:55:18.333228 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":7}
I20260926 06:55:18.333375 16051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:18.323615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:58157 (local address 127.0.191.194:42225)
0926 06:55:18.323755 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:18.323764 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:18.323787 (+ 23us) server_negotiation.cc:407] Connection header received
0926 06:55:18.324179 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:18.324181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:18.324220 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:18.324375 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:18.325077 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:18.326093 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:18.327041 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:18.327445 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:18.327559 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:18.330109 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:18.330112 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:18.330113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:18.330122 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:18.331725 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:18.332187 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:18.332191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:18.332191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:18.332278 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:18.332540 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:18.332541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:18.332541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:18.332754 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:18.332839 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:18.333116 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 06:55:18.333258 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":55,"threads_started":1}
I20260926 06:55:18.333611 16049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:18.323463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:42225 (local address 127.0.191.193:58157)
0926 06:55:18.323573 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:18.323583 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:18.323639 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:18.324412 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:18.324414 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:18.324749 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:18.324920 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:18.324925 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:18.326194 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:18.326194 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:18.326945 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:18.326948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:18.327537 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:18.327538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:18.327638 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:18.328250 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:18.328262 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:18.329995 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:18.331834 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:18.331835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:18.331835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:18.332105 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:18.332355 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:18.332356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:18.332356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:18.332465 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:18.332835 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:18.332836 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:18.332948 (+ 112us) client_negotiation.cc:769] Sending connection context
0926 06:55:18.333046 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 06:55:18.333303 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":189,"thread_start_us":48,"threads_started":1}
W20260926 06:55:18.334333 15663 leader_election.cc:341] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225): Not found: Tablet not found: 6530499a14844ce68f97c23d46d425e3
W20260926 06:55:18.334547 15664 leader_election.cc:341] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645): Not found: Tablet not found: 6530499a14844ce68f97c23d46d425e3
I20260926 06:55:18.334594 15664 leader_election.cc:302] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa0e00f504e64287b73bf26a8f445232; no voters: 2185dc0f2d344beb82881db37c91d88b, 90b4d59ee0484dada801834f1680ad58
I20260926 06:55:18.334693 16048 raft_consensus.cc:2726] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:55:19.121932 16052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:19.121739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:51745)
0926 06:55:19.121852 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:19.121887 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260926 06:55:19.182390 16053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:19.182194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:38683)
0926 06:55:19.182316 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:19.182350 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260926 06:55:19.301494 16054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:19.301303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:59757)
0926 06:55:19.301421 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:19.301458 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":55,"threads_started":1}
I20260926 06:55:20.032051 16055 raft_consensus.cc:482] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:55:20.032142 16055 raft_consensus.cc:504] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:20.032351 16055 leader_election.cc:288] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225), 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645)
W20260926 06:55:20.032735 15664 leader_election.cc:341] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645): Not found: Tablet not found: 6530499a14844ce68f97c23d46d425e3
W20260926 06:55:20.032819 15663 leader_election.cc:341] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225): Not found: Tablet not found: 6530499a14844ce68f97c23d46d425e3
I20260926 06:55:20.032915 15663 leader_election.cc:302] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa0e00f504e64287b73bf26a8f445232; no voters: 2185dc0f2d344beb82881db37c91d88b, 90b4d59ee0484dada801834f1680ad58
I20260926 06:55:20.033022 16055 raft_consensus.cc:2726] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:55:20.122610 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:20.122406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:57773)
0926 06:55:20.122527 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:20.122564 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260926 06:55:20.183043 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:20.182801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:59835)
0926 06:55:20.182956 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:20.183006 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":56,"threads_started":1}
I20260926 06:55:20.302112 16058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:20.301925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:53203)
0926 06:55:20.302045 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:20.302079 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260926 06:55:21.123358 16059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:21.123140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:54551)
0926 06:55:21.123275 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:21.123309 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260926 06:55:21.183683 16060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:21.183498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:44073)
0926 06:55:21.183613 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:21.183648 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260926 06:55:21.302737 16061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:21.302550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:46183)
0926 06:55:21.302668 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:21.302704 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260926 06:55:22.124238 16062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:22.123953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:38137)
0926 06:55:22.124103 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:22.124172 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":60,"threads_started":1}
I20260926 06:55:22.184300 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:22.184081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:43835)
0926 06:55:22.184213 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:22.184258 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260926 06:55:22.303382 16064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:22.303158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:44761)
0926 06:55:22.303310 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:22.303343 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260926 06:55:23.049023 16065 raft_consensus.cc:482] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:55:23.049113 16065 raft_consensus.cc:504] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:23.049304 16065 leader_election.cc:288] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225), 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645)
W20260926 06:55:23.049685 15664 leader_election.cc:341] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645): Not found: Tablet not found: 6530499a14844ce68f97c23d46d425e3
W20260926 06:55:23.049768 15663 leader_election.cc:341] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225): Not found: Tablet not found: 6530499a14844ce68f97c23d46d425e3
I20260926 06:55:23.049832 15663 leader_election.cc:302] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa0e00f504e64287b73bf26a8f445232; no voters: 2185dc0f2d344beb82881db37c91d88b, 90b4d59ee0484dada801834f1680ad58
I20260926 06:55:23.049957 16065 raft_consensus.cc:2726] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:55:23.124958 16066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:23.124716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:37037)
0926 06:55:23.124861 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:23.124914 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260926 06:55:23.184907 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:23.184729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:42229)
0926 06:55:23.184834 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:23.184869 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
I20260926 06:55:23.304163 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:23.303963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:41759)
0926 06:55:23.304080 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:23.304120 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260926 06:55:24.125641 16069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:24.125442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:54421)
0926 06:55:24.125562 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:24.125597 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260926 06:55:24.185566 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:24.185318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:48809)
0926 06:55:24.185494 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:24.185528 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":106,"threads_started":1}
I20260926 06:55:24.304811 16071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:24.304634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:38209)
0926 06:55:24.304741 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:24.304774 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":56,"threads_started":1}
I20260926 06:55:25.126317 16072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:25.126117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:53919)
0926 06:55:25.126235 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:25.126273 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260926 06:55:25.186151 16073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:25.185955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:41219)
0926 06:55:25.186076 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:25.186111 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260926 06:55:25.305642 16074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:25.305435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:55035)
0926 06:55:25.305561 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:25.305600 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260926 06:55:26.127117 16075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:26.126891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:37511)
0926 06:55:26.127031 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:26.127069 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260926 06:55:26.186856 16076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:26.186656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:49697)
0926 06:55:26.186776 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:26.186814 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
I20260926 06:55:26.306277 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:26.306082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:54413)
0926 06:55:26.306203 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:26.306236 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260926 06:55:27.127930 16078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:27.127721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:57113)
0926 06:55:27.127846 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:27.127885 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260926 06:55:27.187462 16079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:27.187256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:55477)
0926 06:55:27.187386 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:27.187422 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
I20260926 06:55:27.306945 16080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:27.306704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:37347)
0926 06:55:27.306853 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:27.306893 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":75,"threads_started":1}
I20260926 06:55:27.479792 16081 raft_consensus.cc:482] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:55:27.479887 16081 raft_consensus.cc:504] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2185dc0f2d344beb82881db37c91d88b" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42225 } } peers { permanent_uuid: "90b4d59ee0484dada801834f1680ad58" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35645 } } peers { permanent_uuid: "aa0e00f504e64287b73bf26a8f445232" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39687 } }
I20260926 06:55:27.480051 16081 leader_election.cc:288] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225), 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645)
W20260926 06:55:27.480476 15664 leader_election.cc:341] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90b4d59ee0484dada801834f1680ad58 (127.0.191.195:35645): Not found: Tablet not found: 6530499a14844ce68f97c23d46d425e3
W20260926 06:55:27.480569 15663 leader_election.cc:341] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2185dc0f2d344beb82881db37c91d88b (127.0.191.194:42225): Not found: Tablet not found: 6530499a14844ce68f97c23d46d425e3
I20260926 06:55:27.480615 15663 leader_election.cc:302] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa0e00f504e64287b73bf26a8f445232; no voters: 2185dc0f2d344beb82881db37c91d88b, 90b4d59ee0484dada801834f1680ad58
I20260926 06:55:27.480719 16081 raft_consensus.cc:2726] T 6530499a14844ce68f97c23d46d425e3 P aa0e00f504e64287b73bf26a8f445232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:55:28.128641 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:28.128422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:39419)
0926 06:55:28.128556 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:28.128597 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
I20260926 06:55:28.188103 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:28.187909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:47477)
0926 06:55:28.188024 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:28.188059 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260926 06:55:28.307650 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:28.307430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:43349)
0926 06:55:28.307566 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:28.307607 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260926 06:55:29.129603 16085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:29.129234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:49147)
0926 06:55:29.129494 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:29.129550 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":81,"threads_started":1}
I20260926 06:55:29.188732 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:29.188521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:38469)
0926 06:55:29.188646 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:29.188687 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260926 06:55:29.308408 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:29.308141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:59031)
0926 06:55:29.308307 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:29.308352 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":60,"threads_started":1}
I20260926 06:55:30.130479 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:30.130249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:39443)
0926 06:55:30.130399 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:30.130435 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260926 06:55:30.189500 16089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:30.189213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:53509)
0926 06:55:30.189340 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:30.189448 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260926 06:55:30.309477 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:30.309183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:57325)
0926 06:55:30.309327 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:30.309421 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260926 06:55:31.131119 16091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:31.130914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:55403)
0926 06:55:31.131038 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:31.131073 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260926 06:55:31.190160 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:31.189953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:44993)
0926 06:55:31.190079 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:31.190115 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260926 06:55:31.310133 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:31.309913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:51139)
0926 06:55:31.310055 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:31.310089 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260926 06:55:32.131893 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:32.131662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.193:60155)
0926 06:55:32.131808 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:32.131849 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":62,"threads_started":1}
I20260926 06:55:32.191033 16095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:32.190760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.194:38697)
0926 06:55:32.190944 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:32.190987 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":87,"threads_started":1}
I20260926 06:55:32.310863 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:32.310647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:39441 (local address 127.0.191.195:52713)
0926 06:55:32.310775 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:32.310818 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:39441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260926 06:55:32.318408 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 15616
I20260926 06:55:32.323524 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 15780
I20260926 06:55:32.328159 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 15912
2026-09-26T06:55:32Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18571 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:55:32 dist-test-slave-ws74 krb5kdc[16099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:55:32 dist-test-slave-ws74 krb5kdc[16099](info): set up 2 sockets
Sep 26 06:55:32 dist-test-slave-ws74 krb5kdc[16099](info): commencing operation
krb5kdc: starting...
W20260926 06:55:33.373551 767 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:55:33 dist-test-slave-ws74 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:55:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:55:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:55:33.492679 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:36031
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:36031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:33.576565 16112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:33.576804 16112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:33.576843 16112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:33.578078 16112 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 06:55:33.578142 16112 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 06:55:33.578162 16112 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 06:55:33.578181 16112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:55:33.578234 16112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:33.578263 16112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:33.578291 16112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:55:33.578321 16112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:55:33.579804 16112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:36031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:36031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:33.580096 16112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:33.580327 16112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:33.585467 16120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:33.585500 16117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:33.585485 16118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:33.585588 16112 server_base.cc:1034] running on GCE node
I20260926 06:55:33.585997 16112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:33.586287 16112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:33.587443 16112 hybrid_clock.cc:648] HybridClock initialized: now 1790405733587453 us; error 48 us; skew 500 ppm
Sep 26 06:55:33 dist-test-slave-ws74 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405733, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:33.595410 16112 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:55:33.595835 16112 webserver.cc:460] Webserver started at http://127.0.191.254:39555/ using document root <none> and password file <none>
I20260926 06:55:33.596076 16112 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:33.596144 16112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:33.596293 16112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:55:33.597357 16112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "f17c7b1a08bc494dbea725394e0f83c4"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "216bf1c1a87fd4e955d769bd8b69af03"
server_key_iv: "a8a6d81c432f0951f6d364a250aca844"
server_key_version: "encryptionkey@0"
I20260926 06:55:33.597810 16112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "f17c7b1a08bc494dbea725394e0f83c4"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "216bf1c1a87fd4e955d769bd8b69af03"
server_key_iv: "a8a6d81c432f0951f6d364a250aca844"
server_key_version: "encryptionkey@0"
I20260926 06:55:33.599115 16112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:55:33.599829 16127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:33.600003 16112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:55:33.600099 16112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "f17c7b1a08bc494dbea725394e0f83c4"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "216bf1c1a87fd4e955d769bd8b69af03"
server_key_iv: "a8a6d81c432f0951f6d364a250aca844"
server_key_version: "encryptionkey@0"
I20260926 06:55:33.600185 16112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:33.637622 16112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:33.639671 16112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:33.639842 16112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:33.644285 16112 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:36031
I20260926 06:55:33.644323 16179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:36031 every 8 connection(s)
I20260926 06:55:33.644690 16112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:55:33.645592 16180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:33.647686 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 16112
I20260926 06:55:33.647814 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:55:33.647876 16180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Bootstrap starting.
I20260926 06:55:33.648064 767 external_mini_cluster.cc:1428] Setting key 0b41dbeb8255fec37ffd4397a1438529
I20260926 06:55:33.648593 16180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:33.648924 16180 log.cc:824] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:33.649645 16180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: No bootstrap required, opened a new log
I20260926 06:55:33.651397 16180 raft_consensus.cc:348] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } }
I20260926 06:55:33.651551 16180 raft_consensus.cc:374] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:33.651595 16180 raft_consensus.cc:729] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f17c7b1a08bc494dbea725394e0f83c4, State: Initialized, Role: FOLLOWER
I20260926 06:55:33.651724 16180 consensus_queue.cc:260] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } }
I20260926 06:55:33.651818 16180 raft_consensus.cc:388] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:55:33.651868 16180 raft_consensus.cc:482] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:55:33.651917 16180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:33.652432 16180 raft_consensus.cc:504] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } }
I20260926 06:55:33.652552 16180 leader_election.cc:302] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f17c7b1a08bc494dbea725394e0f83c4; no voters:
I20260926 06:55:33.652730 16180 leader_election.cc:288] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:55:33.652753 16185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:55:33.652868 16185 raft_consensus.cc:686] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 1 LEADER]: Becoming Leader. State: Replica: f17c7b1a08bc494dbea725394e0f83c4, State: Running, Role: LEADER
I20260926 06:55:33.653013 16180 sys_catalog.cc:564] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:55:33.653004 16185 consensus_queue.cc:237] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } }
I20260926 06:55:33.653486 16187 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f17c7b1a08bc494dbea725394e0f83c4. Latest consensus state: current_term: 1 leader_uuid: "f17c7b1a08bc494dbea725394e0f83c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } } }
I20260926 06:55:33.653653 16187 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:33.654387 16186 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f17c7b1a08bc494dbea725394e0f83c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } } }
I20260926 06:55:33.654471 16186 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: This master's current role is: LEADER
Sep 26 06:55:33 dist-test-slave-ws74 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
W20260926 06:55:33.655506 16201 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:55:33.655565 16201 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:55:33.655689 16194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:55:33.655918 16194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:55:33.657675 16194 catalog_manager.cc:1347] Generated new cluster ID: ae3a63648228414ca0f2752415eb63eb
I20260926 06:55:33.657729 16194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:55:33.658794 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:33.648595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36246 (local address 127.0.191.254:36031)
0926 06:55:33.648792 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:33.648800 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:33.648824 (+ 24us) server_negotiation.cc:407] Connection header received
0926 06:55:33.649252 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:33.649258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:33.649301 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:33.649528 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:33.650210 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:33.651071 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:33.651963 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:33.652377 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:33.652510 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:33.655220 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:33.655224 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:33.655224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:33.655238 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:33.657192 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:33.657644 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:33.657646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:33.657647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:33.657759 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:33.657968 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:33.657969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:33.657969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:33.658195 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:33.658345 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:33.658472 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 06:55:33.658609 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":74,"threads_started":1}
I20260926 06:55:33.678249 16194 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:55:33.678792 16194 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:55:33.685031 16194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Generated new TSK 0
I20260926 06:55:33.685236 16194 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:55:33.714228 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:36031
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:33.794718 16206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:33.794921 16206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:33.794945 16206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:33.796067 16206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:33.796108 16206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:33.796161 16206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:55:33.797616 16206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:36031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:33.797801 16206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:33.797973 16206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:33.802973 16211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:33.803009 16214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:33.803049 16206 server_base.cc:1034] running on GCE node
W20260926 06:55:33.803066 16212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:33.803491 16206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:33.803767 16206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:33.804936 16206 hybrid_clock.cc:648] HybridClock initialized: now 1790405733804931 us; error 38 us; skew 500 ppm
Sep 26 06:55:33 dist-test-slave-ws74 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405733, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:33.807571 16206 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:55:33.807941 16206 webserver.cc:460] Webserver started at http://127.0.191.193:33807/ using document root <none> and password file <none>
I20260926 06:55:33.808120 16206 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:33.808162 16206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:33.808296 16206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:55:33.809305 16206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "04a5c1e9863e481387554126e1f6383a"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "ce83d71e0949faf8d82ad6a4f20d4e21"
server_key_iv: "d68c434d4cd123eadac903539e149abd"
server_key_version: "encryptionkey@0"
I20260926 06:55:33.809732 16206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "04a5c1e9863e481387554126e1f6383a"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "ce83d71e0949faf8d82ad6a4f20d4e21"
server_key_iv: "d68c434d4cd123eadac903539e149abd"
server_key_version: "encryptionkey@0"
I20260926 06:55:33.811010 16206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:33.811712 16221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:33.811875 16206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:55:33.811954 16206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "04a5c1e9863e481387554126e1f6383a"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "ce83d71e0949faf8d82ad6a4f20d4e21"
server_key_iv: "d68c434d4cd123eadac903539e149abd"
server_key_version: "encryptionkey@0"
I20260926 06:55:33.812004 16206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:33.822365 16206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:33.824259 16206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:33.824364 16206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:33.824538 16206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:33.824831 16206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:33.824862 16206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:33.824882 16206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:33.824930 16206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:33.830495 16206 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:33491
I20260926 06:55:33.830572 16334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:33491 every 8 connection(s)
I20260926 06:55:33.830866 16206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:55:33 dist-test-slave-ws74 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405733, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:33.838752 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 16206
I20260926 06:55:33.838882 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:55:33.839115 767 external_mini_cluster.cc:1428] Setting key e4a9fd342363d0d2f200fc8ed827640b
I20260926 06:55:33.839401 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:33.831596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:34303 (local address 127.0.191.254:36031)
0926 06:55:33.831669 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:33.831672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:33.831810 (+ 138us) server_negotiation.cc:407] Connection header received
0926 06:55:33.832250 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:33.832251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:33.832260 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:33.832310 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:33.833169 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:33.833748 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:33.834664 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:33.834853 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:33.834905 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:33.836903 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:33.836904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:33.836904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:33.836909 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:33.838089 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:33.838443 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:33.838444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:33.838444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:33.838493 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:33.838707 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:33.838708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:33.838708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:33.838854 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:33.838930 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:33.839210 (+ 280us) server_negotiation.cc:299] Negotiation successful
0926 06:55:33.839312 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 06:55:33.839650 16337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:33.831539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:34303)
0926 06:55:33.831695 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:33.831728 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:33.832174 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:33.832366 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:33.832369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:33.832914 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:33.833094 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:33.833098 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:33.833822 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:33.833823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:33.834596 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:33.834598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:33.834921 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:33.834922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:33.835019 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:33.835539 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:33.835547 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:33.836821 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:33.838156 (+ 1335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:33.838162 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:33.838163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:33.838380 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:33.838550 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:33.838550 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:33.838551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:33.838641 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:33.838920 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:33.838925 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:33.839076 (+ 151us) client_negotiation.cc:769] Sending connection context
0926 06:55:33.839160 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 06:55:33.839384 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260926 06:55:33.841285 16335 heartbeater.cc:347] Connected to a master server at 127.0.191.254:36031
I20260926 06:55:33.841365 16335 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:33.841586 16335 heartbeater.cc:511] Master 127.0.191.254:36031 requested a full tablet report, sending...
I20260926 06:55:33.842127 16144 ts_manager.cc:194] Registered new tserver with Master: 04a5c1e9863e481387554126e1f6383a (127.0.191.193:33491)
I20260926 06:55:33.842728 16144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:34303
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:55:33.863358 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:36031
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:33.944229 16340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:33.944501 16340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:33.944541 16340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:33.945699 16340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:33.945772 16340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:33.945816 16340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:55:33.947235 16340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:36031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:33.947502 16340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:33.947726 16340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:33.952911 16348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:33.952926 16346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:33.953039 16345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:33.953106 16340 server_base.cc:1034] running on GCE node
I20260926 06:55:33.953502 16340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:33.953786 16340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:33.954947 16340 hybrid_clock.cc:648] HybridClock initialized: now 1790405733954939 us; error 32 us; skew 500 ppm
Sep 26 06:55:33 dist-test-slave-ws74 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405733, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:33.957468 16340 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:55:33.957933 16340 webserver.cc:460] Webserver started at http://127.0.191.194:39213/ using document root <none> and password file <none>
I20260926 06:55:33.958151 16340 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:33.958217 16340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:33.958355 16340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:55:33.959626 16340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "00a0eeeabef14875aee59035e4cf1526"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "c70632b8614752ab7f77c895ff4dc648"
server_key_iv: "5fc9e2b7616dbb927bfc15c121ed843c"
server_key_version: "encryptionkey@0"
I20260926 06:55:33.960038 16340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "00a0eeeabef14875aee59035e4cf1526"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "c70632b8614752ab7f77c895ff4dc648"
server_key_iv: "5fc9e2b7616dbb927bfc15c121ed843c"
server_key_version: "encryptionkey@0"
I20260926 06:55:33.961587 16340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:55:33.962344 16355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:33.962509 16340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:55:33.962616 16340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "00a0eeeabef14875aee59035e4cf1526"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "c70632b8614752ab7f77c895ff4dc648"
server_key_iv: "5fc9e2b7616dbb927bfc15c121ed843c"
server_key_version: "encryptionkey@0"
I20260926 06:55:33.962692 16340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:33.976569 16340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:33.978922 16340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:33.979082 16340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:33.979303 16340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:33.979592 16340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:33.979645 16340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:33.979682 16340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:33.979713 16340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:33.985586 16340 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:42177
I20260926 06:55:33.985638 16468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:42177 every 8 connection(s)
I20260926 06:55:33.985957 16340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:55:33.987871 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 16340
I20260926 06:55:33.987967 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:55:33.988164 767 external_mini_cluster.cc:1428] Setting key ed2c18924b6d7881555de2bfd567ec62
Sep 26 06:55:33 dist-test-slave-ws74 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405733, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
I20260926 06:55:33.995807 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:33.986674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:34385 (local address 127.0.191.254:36031)
0926 06:55:33.986716 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:33.986718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:33.986921 (+ 203us) server_negotiation.cc:407] Connection header received
0926 06:55:33.987402 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:33.987403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:33.987411 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:33.987469 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:33.988557 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:33.989158 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:33.990220 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:33.990432 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:33.990518 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:33.992907 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:33.992908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:33.992909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:33.992916 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:33.994378 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:33.994781 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:33.994782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:33.994782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:33.994849 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:33.995100 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:33.995101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:33.995102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:33.995218 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:33.995293 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:33.995601 (+ 308us) server_negotiation.cc:299] Negotiation successful
0926 06:55:33.995681 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:55:33.995836 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:33.986632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:34385)
0926 06:55:33.986794 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:33.986835 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:33.987308 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:33.987520 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:33.987523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:33.988147 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:33.988454 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:33.988460 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:33.989254 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:33.989255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:33.990116 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:33.990118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:33.990529 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:33.990530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:33.990664 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:33.991234 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:33.991242 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:33.992805 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:33.994479 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:33.994486 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:33.994488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:33.994706 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:33.994923 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:33.994924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:33.994924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:33.995032 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:33.995285 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:33.995291 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:33.995432 (+ 141us) client_negotiation.cc:769] Sending connection context
0926 06:55:33.995544 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 06:55:33.995687 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260926 06:55:33.996348 16469 heartbeater.cc:347] Connected to a master server at 127.0.191.254:36031
I20260926 06:55:33.996502 16469 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:33.996694 16469 heartbeater.cc:511] Master 127.0.191.254:36031 requested a full tablet report, sending...
I20260926 06:55:33.997094 16144 ts_manager.cc:194] Registered new tserver with Master: 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177)
I20260926 06:55:33.997534 16144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:34385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:55:34.011795 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:36031
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:34.091260 16474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:34.091456 16474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:34.091478 16474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:34.092523 16474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:34.092555 16474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:34.092581 16474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:55:34.093892 16474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:36031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:34.094051 16474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:34.094224 16474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:34.098886 16479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:34.098909 16482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:34.098913 16480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:34.099278 16474 server_base.cc:1034] running on GCE node
I20260926 06:55:34.099472 16474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:34.099700 16474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:34.100869 16474 hybrid_clock.cc:648] HybridClock initialized: now 1790405734100853 us; error 23 us; skew 500 ppm
Sep 26 06:55:34 dist-test-slave-ws74 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405734, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:34.103418 16474 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:55:34.103902 16474 webserver.cc:460] Webserver started at http://127.0.191.195:42901/ using document root <none> and password file <none>
I20260926 06:55:34.104182 16474 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:34.104239 16474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:34.104327 16474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:55:34.105250 16474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "460f16e281384d10aa8a29107c8665a0"
format_stamp: "Formatted at 2026-09-26 06:55:34 on dist-test-slave-ws74"
server_key: "542857a999462d40e5a592f1bf2f3f3f"
server_key_iv: "b588878465ed31d923872d6ed1943e58"
server_key_version: "encryptionkey@0"
I20260926 06:55:34.105628 16474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "460f16e281384d10aa8a29107c8665a0"
format_stamp: "Formatted at 2026-09-26 06:55:34 on dist-test-slave-ws74"
server_key: "542857a999462d40e5a592f1bf2f3f3f"
server_key_iv: "b588878465ed31d923872d6ed1943e58"
server_key_version: "encryptionkey@0"
I20260926 06:55:34.106881 16474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:34.107563 16489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:34.107728 16474 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:55:34.107824 16474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "460f16e281384d10aa8a29107c8665a0"
format_stamp: "Formatted at 2026-09-26 06:55:34 on dist-test-slave-ws74"
server_key: "542857a999462d40e5a592f1bf2f3f3f"
server_key_iv: "b588878465ed31d923872d6ed1943e58"
server_key_version: "encryptionkey@0"
I20260926 06:55:34.107903 16474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:34.119455 16474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:34.121737 16474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:34.121898 16474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:34.122115 16474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:34.122416 16474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:34.122470 16474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:34.122509 16474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:34.122536 16474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:34.128191 16474 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:40299
I20260926 06:55:34.128226 16602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:40299 every 8 connection(s)
I20260926 06:55:34.128530 16474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:55:34 dist-test-slave-ws74 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405734, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:34.136246 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 16474
I20260926 06:55:34.136358 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:55:34.136569 767 external_mini_cluster.cc:1428] Setting key 7e027d83b36c076acf8fb8db95051515
I20260926 06:55:34.137058 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.129212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:53575 (local address 127.0.191.254:36031)
0926 06:55:34.129258 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:34.129260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:34.129471 (+ 211us) server_negotiation.cc:407] Connection header received
0926 06:55:34.129913 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:34.129914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:34.129924 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:34.129983 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:34.130842 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.131322 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.132230 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.132409 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.132458 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.134492 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:34.134493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:34.134493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:34.134500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:34.135690 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:34.136033 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:34.136034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:34.136035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:34.136082 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:34.136369 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:34.136370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:34.136370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:34.136510 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:34.136564 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:34.136886 (+ 322us) server_negotiation.cc:299] Negotiation successful
0926 06:55:34.136972 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:55:34.137205 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.129168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:53575)
0926 06:55:34.129320 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:34.129358 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:34.129826 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:34.130053 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:34.130057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:34.130590 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:34.130767 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:34.130771 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.131394 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.131394 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:34.132162 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:34.132163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.132477 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.132477 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:34.132569 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.133041 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:34.133056 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:34.134404 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:34.135756 (+ 1352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:34.135761 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:34.135763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:34.135971 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:34.136145 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:34.136145 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:34.136146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:34.136284 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:34.136576 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:34.136582 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:34.136738 (+ 156us) client_negotiation.cc:769] Sending connection context
0926 06:55:34.136853 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 06:55:34.136996 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260926 06:55:34.137678 16603 heartbeater.cc:347] Connected to a master server at 127.0.191.254:36031
I20260926 06:55:34.137749 16603 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:34.137888 16603 heartbeater.cc:511] Master 127.0.191.254:36031 requested a full tablet report, sending...
I20260926 06:55:34.138183 16144 ts_manager.cc:194] Registered new tserver with Master: 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299)
I20260926 06:55:34.138540 16144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:53575
I20260926 06:55:34.148386 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:55:34.155676 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.149655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36252 (local address 127.0.191.254:36031)
0926 06:55:34.149707 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:34.149708 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:34.149849 (+ 141us) server_negotiation.cc:407] Connection header received
0926 06:55:34.149934 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:34.149935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:34.149945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:34.149992 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:34.150532 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.151004 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.151709 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.151913 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.151986 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.153233 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:34.153234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:34.153234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:34.153241 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:34.154672 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:34.154999 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:34.154999 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:34.155000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:34.155045 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:34.155251 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:34.155251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:34.155251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:34.155372 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:34.155446 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:34.155561 (+ 115us) server_negotiation.cc:299] Negotiation successful
0926 06:55:34.155618 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:55:34.157069 16144 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:55:34.158677 16144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:55:34.164836 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.160939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:33491 (local address 127.0.0.1:55692)
0926 06:55:34.161120 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:34.161129 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:34.161171 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:34.161668 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:34.161669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:34.161671 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:34.161710 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:34.161712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.162950 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.162951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:34.163688 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:34.163690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.164550 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.164551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:34.164623 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.164637 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 06:55:34.164712 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:55:34.164759 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":41,"threads_started":1}
I20260926 06:55:34.164865 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.161049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55692 (local address 127.0.191.193:33491)
0926 06:55:34.161500 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:34.161504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:34.161514 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:55:34.161548 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:34.161550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:34.161567 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:34.161780 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:34.161852 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.162855 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.163788 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.164464 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.164538 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.164592 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:34.164738 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 06:55:34.164781 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":86,"threads_started":1}
I20260926 06:55:34.165138 16612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.160726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:42177 (local address 127.0.0.1:56474)
0926 06:55:34.161051 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:34.161080 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:34.161150 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:34.161704 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:34.161705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:34.161707 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:34.161745 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:34.161746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.162605 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.162606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:34.163927 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:34.163929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.164849 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.164850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:34.164929 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.164934 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:55:34.164988 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 06:55:34.165061 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":30,"thread_start_us":50,"threads_started":1}
I20260926 06:55:34.165244 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.160715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:40299 (local address 127.0.0.1:38558)
0926 06:55:34.161039 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:34.161079 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:34.161149 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:34.161395 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:34.161399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:34.161401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:34.161466 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:34.161468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.162735 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.162736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:34.164134 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:34.164135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.164989 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.164990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:34.165062 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.165065 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:55:34.165117 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 06:55:34.165155 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":8,"thread_start_us":56,"threads_started":1}
I20260926 06:55:34.165282 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.160888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38558 (local address 127.0.191.195:40299)
0926 06:55:34.161078 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:34.161084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:34.161224 (+ 140us) server_negotiation.cc:407] Connection header received
0926 06:55:34.161270 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:34.161272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:34.161284 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:34.161340 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:34.161530 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.162641 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.164218 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.164823 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.164887 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.164949 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:34.165146 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 06:55:34.165192 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":86,"threads_started":1}
I20260926 06:55:34.165412 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.160821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56474 (local address 127.0.191.194:42177)
0926 06:55:34.161289 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:34.161292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:34.161300 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:55:34.161365 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:34.161367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:34.161405 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:34.161624 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:34.161837 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.162508 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.164007 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.164679 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.164764 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.164815 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:34.165233 (+ 418us) server_negotiation.cc:299] Negotiation successful
0926 06:55:34.165279 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":96,"threads_started":1}
I20260926 06:55:34.166075 16537 tablet_service.cc:1467] Processing CreateTablet for tablet b7bcef7a74a5426683843a0aaf39c1bb (DEFAULT_TABLE table=test-table [id=adae5a31ea554cc292d031c9e0424e27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:34.166226 16269 tablet_service.cc:1467] Processing CreateTablet for tablet b7bcef7a74a5426683843a0aaf39c1bb (DEFAULT_TABLE table=test-table [id=adae5a31ea554cc292d031c9e0424e27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:34.166319 16403 tablet_service.cc:1467] Processing CreateTablet for tablet b7bcef7a74a5426683843a0aaf39c1bb (DEFAULT_TABLE table=test-table [id=adae5a31ea554cc292d031c9e0424e27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:34.166421 16537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7bcef7a74a5426683843a0aaf39c1bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:34.166565 16269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7bcef7a74a5426683843a0aaf39c1bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:34.166594 16403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7bcef7a74a5426683843a0aaf39c1bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:34.169073 16617 tablet_bootstrap.cc:492] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0: Bootstrap starting.
I20260926 06:55:34.169718 16618 tablet_bootstrap.cc:492] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526: Bootstrap starting.
I20260926 06:55:34.169843 16617 tablet_bootstrap.cc:654] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:34.169875 16619 tablet_bootstrap.cc:492] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Bootstrap starting.
I20260926 06:55:34.170406 16619 tablet_bootstrap.cc:654] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:34.170440 16618 tablet_bootstrap.cc:654] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:34.170446 16617 log.cc:824] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:34.170953 16618 log.cc:824] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:34.170979 16619 log.cc:824] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:34.171262 16617 tablet_bootstrap.cc:492] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0: No bootstrap required, opened a new log
I20260926 06:55:34.171337 16617 ts_tablet_manager.cc:1397] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:55:34.171693 16618 tablet_bootstrap.cc:492] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526: No bootstrap required, opened a new log
I20260926 06:55:34.171693 16619 tablet_bootstrap.cc:492] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: No bootstrap required, opened a new log
I20260926 06:55:34.171762 16618 ts_tablet_manager.cc:1397] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:55:34.171762 16619 ts_tablet_manager.cc:1397] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:55:34.172932 16617 raft_consensus.cc:348] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:34.173053 16617 raft_consensus.cc:374] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:34.173074 16617 raft_consensus.cc:729] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460f16e281384d10aa8a29107c8665a0, State: Initialized, Role: FOLLOWER
I20260926 06:55:34.173187 16617 consensus_queue.cc:260] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:34.173470 16617 ts_tablet_manager.cc:1428] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:55:34.173487 16603 heartbeater.cc:503] Master 127.0.191.254:36031 was elected leader, sending a full tablet report...
I20260926 06:55:34.173679 16618 raft_consensus.cc:348] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:34.173838 16618 raft_consensus.cc:374] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:34.173872 16618 raft_consensus.cc:729] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a0eeeabef14875aee59035e4cf1526, State: Initialized, Role: FOLLOWER
I20260926 06:55:34.173975 16618 consensus_queue.cc:260] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:34.173993 16619 raft_consensus.cc:348] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:34.174150 16619 raft_consensus.cc:374] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:34.174185 16619 raft_consensus.cc:729] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a5c1e9863e481387554126e1f6383a, State: Initialized, Role: FOLLOWER
I20260926 06:55:34.174242 16618 ts_tablet_manager.cc:1428] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:55:34.174293 16469 heartbeater.cc:503] Master 127.0.191.254:36031 was elected leader, sending a full tablet report...
I20260926 06:55:34.174321 16619 consensus_queue.cc:260] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:34.174572 16619 ts_tablet_manager.cc:1428] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:55:34.174695 16335 heartbeater.cc:503] Master 127.0.191.254:36031 was elected leader, sending a full tablet report...
I20260926 06:55:34.228818 16625 raft_consensus.cc:482] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:55:34.228960 16625 raft_consensus.cc:504] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:34.229279 16625 leader_election.cc:288] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299), 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177)
I20260926 06:55:34.232044 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.229576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:35591 (local address 127.0.191.194:42177)
0926 06:55:34.229628 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:34.229630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:34.229638 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:55:34.229672 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:34.229673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:34.229682 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:34.229763 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:34.229925 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.230398 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.231329 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.231702 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.231787 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.231796 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:34.231943 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 06:55:34.231980 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 06:55:34.232045 16337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.229485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:42177 (local address 127.0.191.193:35591)
0926 06:55:34.229524 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:34.229536 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:34.229578 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:34.229769 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:34.229770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:34.229772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:34.229846 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:34.229848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.230486 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.230487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:34.231235 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:34.231236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.231782 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.231783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:34.231866 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.231869 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:55:34.231936 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 06:55:34.231975 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":25}
I20260926 06:55:34.232170 16626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.229485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:40299 (local address 127.0.191.193:34309)
0926 06:55:34.229693 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:34.229708 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:34.229770 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:34.229917 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:34.229918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:34.229920 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:34.229988 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:34.229990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.230601 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.230602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:34.231347 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:34.231349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.231899 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.231900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:34.231971 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.231975 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:55:34.232049 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 06:55:34.232090 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":44,"thread_start_us":105,"threads_started":1}
I20260926 06:55:34.232157 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.229574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:34309 (local address 127.0.191.195:40299)
0926 06:55:34.229624 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:34.229626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:34.229770 (+ 144us) server_negotiation.cc:407] Connection header received
0926 06:55:34.229831 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:34.229832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:34.229852 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:34.229912 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:34.230066 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.230521 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.231436 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.231816 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.231899 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.231908 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:34.232054 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 06:55:34.232091 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:55:34.232426 16423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7bcef7a74a5426683843a0aaf39c1bb" candidate_uuid: "04a5c1e9863e481387554126e1f6383a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a0eeeabef14875aee59035e4cf1526" is_pre_election: true
I20260926 06:55:34.232507 16557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7bcef7a74a5426683843a0aaf39c1bb" candidate_uuid: "04a5c1e9863e481387554126e1f6383a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460f16e281384d10aa8a29107c8665a0" is_pre_election: true
I20260926 06:55:34.232591 16423 raft_consensus.cc:2454] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04a5c1e9863e481387554126e1f6383a in term 0.
I20260926 06:55:34.232654 16557 raft_consensus.cc:2454] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04a5c1e9863e481387554126e1f6383a in term 0.
I20260926 06:55:34.232821 16224 leader_election.cc:302] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00a0eeeabef14875aee59035e4cf1526, 04a5c1e9863e481387554126e1f6383a; no voters:
I20260926 06:55:34.233001 16625 raft_consensus.cc:2781] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:55:34.233072 16625 raft_consensus.cc:482] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:55:34.233103 16625 raft_consensus.cc:3037] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:34.233851 16625 raft_consensus.cc:504] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:34.234019 16625 leader_election.cc:288] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 1 election: Requested vote from peers 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299), 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177)
I20260926 06:55:34.234145 16557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7bcef7a74a5426683843a0aaf39c1bb" candidate_uuid: "04a5c1e9863e481387554126e1f6383a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460f16e281384d10aa8a29107c8665a0"
I20260926 06:55:34.234181 16423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7bcef7a74a5426683843a0aaf39c1bb" candidate_uuid: "04a5c1e9863e481387554126e1f6383a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a0eeeabef14875aee59035e4cf1526"
I20260926 06:55:34.234220 16557 raft_consensus.cc:3037] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:34.234280 16423 raft_consensus.cc:3037] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:34.234969 16557 raft_consensus.cc:2454] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04a5c1e9863e481387554126e1f6383a in term 1.
I20260926 06:55:34.234979 16423 raft_consensus.cc:2454] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04a5c1e9863e481387554126e1f6383a in term 1.
I20260926 06:55:34.235137 16223 leader_election.cc:302] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a5c1e9863e481387554126e1f6383a, 460f16e281384d10aa8a29107c8665a0; no voters:
I20260926 06:55:34.235296 16625 raft_consensus.cc:2781] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:55:34.235476 16625 raft_consensus.cc:686] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 1 LEADER]: Becoming Leader. State: Replica: 04a5c1e9863e481387554126e1f6383a, State: Running, Role: LEADER
I20260926 06:55:34.235576 16625 consensus_queue.cc:237] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
W20260926 06:55:34.236481 16470 tablet.cc:2369] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:55:34.236358 16143 catalog_manager.cc:5495] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a reported cstate change: term changed from 0 to 1, leader changed from <none> to 04a5c1e9863e481387554126e1f6383a (127.0.191.193). New cstate: current_term: 1 leader_uuid: "04a5c1e9863e481387554126e1f6383a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } health_report { overall_health: HEALTHY } } }
I20260926 06:55:34.255517 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.253180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55704 (local address 127.0.191.193:33491)
0926 06:55:34.253236 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:34.253238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:34.253247 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:55:34.253331 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:34.253332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:34.253347 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:34.253471 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:55:34.253634 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.254092 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.254891 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:34.255071 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:34.255123 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:34.255257 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:55:34.255348 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:55:34.255434 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:34.255448 (+ 14us) server_negotiation.cc:299] Negotiation successful
0926 06:55:34.255464 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:55:34.257227 16557 raft_consensus.cc:1261] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0 [term 1 FOLLOWER]: Refusing update from remote peer 04a5c1e9863e481387554126e1f6383a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:55:34.257556 16423 raft_consensus.cc:1261] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526 [term 1 FOLLOWER]: Refusing update from remote peer 04a5c1e9863e481387554126e1f6383a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:55:34.257552 16625 consensus_queue.cc:1035] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [LEADER]: Connected to new peer: Peer: permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:55:34.257820 16625 consensus_queue.cc:1035] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:55:34.260061 16631 mvcc.cc:205] Tried to move back new op lower bound from 7333501887515123712 to 7333501887429464064. Current Snapshot: MvccSnapshot[applied={T|T < 7333501887515123712}]
I20260926 06:55:34.261646 16634 mvcc.cc:205] Tried to move back new op lower bound from 7333501887515123712 to 7333501887429464064. Current Snapshot: MvccSnapshot[applied={T|T < 7333501887515123712}]
I20260926 06:55:34.261811 16143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:55:34.261900 16143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:55:34.262598 16143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Sending DeleteTablet for 3 replicas of tablet b7bcef7a74a5426683843a0aaf39c1bb
I20260926 06:55:34.262924 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 16206
I20260926 06:55:34.262934 16269 tablet_service.cc:1514] Processing DeleteTablet for tablet b7bcef7a74a5426683843a0aaf39c1bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:34 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:55692
I20260926 06:55:34.262945 16537 tablet_service.cc:1514] Processing DeleteTablet for tablet b7bcef7a74a5426683843a0aaf39c1bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:34 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:38558
I20260926 06:55:34.263469 16640 tablet_replica.cc:330] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0: stopping tablet replica
I20260926 06:55:34.263556 16640 raft_consensus.cc:2229] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:55:34.263624 16640 raft_consensus.cc:2258] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:55:34.263978 16640 ts_tablet_manager.cc:1905] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:55:34.265328 16640 ts_tablet_manager.cc:1918] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:55:34.265429 16640 log.cc:1196] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/b7bcef7a74a5426683843a0aaf39c1bb
I20260926 06:55:34.265695 16640 ts_tablet_manager.cc:1939] T b7bcef7a74a5426683843a0aaf39c1bb P 460f16e281384d10aa8a29107c8665a0: Deleting consensus metadata
I20260926 06:55:34.266045 16129 catalog_manager.cc:4841] TS 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299): tablet b7bcef7a74a5426683843a0aaf39c1bb (table test-table [id=adae5a31ea554cc292d031c9e0424e27]) successfully deleted
I20260926 06:55:34.266204 16403 tablet_service.cc:1514] Processing DeleteTablet for tablet b7bcef7a74a5426683843a0aaf39c1bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:34 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:56474
I20260926 06:55:34.266402 16641 tablet_replica.cc:330] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526: stopping tablet replica
I20260926 06:55:34.266513 16641 raft_consensus.cc:2229] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:55:34.266607 16641 raft_consensus.cc:2258] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:55:34.266970 16641 ts_tablet_manager.cc:1905] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:55:34.268321 16641 ts_tablet_manager.cc:1918] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:55:34.268399 16641 log.cc:1196] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/b7bcef7a74a5426683843a0aaf39c1bb
I20260926 06:55:34.268644 16641 ts_tablet_manager.cc:1939] T b7bcef7a74a5426683843a0aaf39c1bb P 00a0eeeabef14875aee59035e4cf1526: Deleting consensus metadata
I20260926 06:55:34.269034 16130 catalog_manager.cc:4841] TS 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177): tablet b7bcef7a74a5426683843a0aaf39c1bb (table test-table [id=adae5a31ea554cc292d031c9e0424e27]) successfully deleted
W20260926 06:55:34.269716 16128 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:33491 (error 108)
I20260926 06:55:34.269982 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 16340
I20260926 06:55:34.270159 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:34.270045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:33491 (local address 127.0.0.1:55716)
0926 06:55:34.270091 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:34.270123 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:33491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 06:55:34.270967 16128 catalog_manager.cc:4584] TS 04a5c1e9863e481387554126e1f6383a (127.0.191.193:33491): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b7bcef7a74a5426683843a0aaf39c1bb: Network error: Client connection negotiation failed: client connection to 127.0.191.193:33491: connect: Connection refused (error 111)
I20260926 06:55:34.274904 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 16474
I20260926 06:55:34.279386 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 16112
I20260926 06:55:54.284510 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:36031
--webserver_interface=127.0.191.254
--webserver_port=39555
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:36031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:55:54.363140 16643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:54.363350 16643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:54.363379 16643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:54.364449 16643 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 06:55:54.364483 16643 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 06:55:54.364496 16643 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 06:55:54.364508 16643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:55:54.364521 16643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:54.364534 16643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:54.364547 16643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:55:54.364562 16643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:55:54.365967 16643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:36031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:36031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=39555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:54.366135 16643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:54.366286 16643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:54.374636 16652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:54.374636 16650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:54.374645 16649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:54.374913 16643 server_base.cc:1034] running on GCE node
I20260926 06:55:54.375211 16643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:54.375512 16643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:54.376700 16643 hybrid_clock.cc:648] HybridClock initialized: now 1790405754376680 us; error 36 us; skew 500 ppm
Sep 26 06:55:54 dist-test-slave-ws74 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405754, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:54.383752 16643 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:55:54.384147 16643 webserver.cc:460] Webserver started at http://127.0.191.254:39555/ using document root <none> and password file <none>
I20260926 06:55:54.384353 16643 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:54.384406 16643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:54.385725 16643 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:54.386389 16659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:54.386564 16643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:54.386644 16643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "f17c7b1a08bc494dbea725394e0f83c4"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "216bf1c1a87fd4e955d769bd8b69af03"
server_key_iv: "a8a6d81c432f0951f6d364a250aca844"
server_key_version: "encryptionkey@0"
I20260926 06:55:54.387024 16643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:54.401739 16643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:54.403554 16643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:54.403656 16643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:54.407539 16643 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:36031
I20260926 06:55:54.407588 16711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:36031 every 8 connection(s)
I20260926 06:55:54.407970 16643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:55:54.409418 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 16643
I20260926 06:55:54.409601 16712 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:55:54.409745 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:33491
--local_ip_for_outbound_sockets=127.0.191.193
--tserver_master_addrs=127.0.191.254:36031
--webserver_port=33807
--webserver_interface=127.0.191.193
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:55:54.410669 16712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Bootstrap starting.
I20260926 06:55:54.418942 16712 log.cc:824] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:54.420739 16712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:55:54.421051 16712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Bootstrap complete.
I20260926 06:55:54.423408 16712 raft_consensus.cc:348] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } }
I20260926 06:55:54.423624 16712 raft_consensus.cc:729] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f17c7b1a08bc494dbea725394e0f83c4, State: Initialized, Role: FOLLOWER
I20260926 06:55:54.423794 16712 consensus_queue.cc:260] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } }
I20260926 06:55:54.423856 16712 raft_consensus.cc:388] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:55:54.423908 16712 raft_consensus.cc:482] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:55:54.423956 16712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 1 FOLLOWER]: Advancing to term 2
I20260926 06:55:54.424926 16712 raft_consensus.cc:504] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } }
I20260926 06:55:54.425034 16712 leader_election.cc:302] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f17c7b1a08bc494dbea725394e0f83c4; no voters:
I20260926 06:55:54.425232 16712 leader_election.cc:288] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 06:55:54.425264 16716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 06:55:54.425398 16716 raft_consensus.cc:686] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 2 LEADER]: Becoming Leader. State: Replica: f17c7b1a08bc494dbea725394e0f83c4, State: Running, Role: LEADER
I20260926 06:55:54.425524 16712 sys_catalog.cc:564] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:55:54.425558 16716 consensus_queue.cc:237] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } }
I20260926 06:55:54.425936 16716 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f17c7b1a08bc494dbea725394e0f83c4. Latest consensus state: current_term: 2 leader_uuid: "f17c7b1a08bc494dbea725394e0f83c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } } }
I20260926 06:55:54.425938 16717 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f17c7b1a08bc494dbea725394e0f83c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } } }
I20260926 06:55:54.426124 16716 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:54.426146 16717 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: This master's current role is: LEADER
I20260926 06:55:54.426687 16725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:55:54.427439 16725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:55:54.427544 16725 catalog_manager.cc:1259] Loaded cluster ID: ae3a63648228414ca0f2752415eb63eb
I20260926 06:55:54.427577 16725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:55:54.428687 16725 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:55:54.428876 16725 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Loaded TSK: 0
I20260926 06:55:54.438443 16725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Generated new TSK 1
I20260926 06:55:54.438529 16725 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 06:55:54.491060 16714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:54.491269 16714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:54.491307 16714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:54.492414 16714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:54.492476 16714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:54.492520 16714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:55:54.494050 16714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:33491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=33807
--tserver_master_addrs=127.0.191.254:36031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:54.494285 16714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:54.494513 16714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:54.499464 16742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:54.499473 16739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:54.499564 16740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:54.499831 16714 server_base.cc:1034] running on GCE node
I20260926 06:55:54.500030 16714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:54.500296 16714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:54.501462 16714 hybrid_clock.cc:648] HybridClock initialized: now 1790405754501431 us; error 44 us; skew 500 ppm
Sep 26 06:55:54 dist-test-slave-ws74 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405754, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:54.504554 16714 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:55:54.504982 16714 webserver.cc:460] Webserver started at http://127.0.191.193:33807/ using document root <none> and password file <none>
I20260926 06:55:54.505223 16714 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:54.505312 16714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:54.506728 16714 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:54.507424 16749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:54.507606 16714 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:54.507685 16714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "04a5c1e9863e481387554126e1f6383a"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "ce83d71e0949faf8d82ad6a4f20d4e21"
server_key_iv: "d68c434d4cd123eadac903539e149abd"
server_key_version: "encryptionkey@0"
I20260926 06:55:54.508031 16714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:54.519543 16714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:54.521786 16714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:54.521944 16714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:54.522176 16714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:54.522578 16756 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 06:55:54.524441 16714 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 06:55:54.524504 16714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 06:55:54.524544 16714 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 06:55:54.525143 16714 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 06:55:54.525197 16714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 06:55:54.525267 16756 tablet_bootstrap.cc:492] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Bootstrap starting.
I20260926 06:55:54.532388 16714 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:33491
I20260926 06:55:54.532492 16863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:33491 every 8 connection(s)
I20260926 06:55:54.532792 16714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:55:54.534646 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 16714
I20260926 06:55:54.534958 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:42177
--local_ip_for_outbound_sockets=127.0.191.194
--tserver_master_addrs=127.0.191.254:36031
--webserver_port=39213
--webserver_interface=127.0.191.194
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:55:54.536187 16756 log.cc:824] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:54.539898 16756 tablet_bootstrap.cc:492] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:55:54.540397 16756 tablet_bootstrap.cc:492] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Bootstrap complete.
I20260926 06:55:54.541610 16756 ts_tablet_manager.cc:1397] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.015s
Sep 26 06:55:54 dist-test-slave-ws74 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405754, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:54.542944 16756 raft_consensus.cc:348] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:54.543306 16756 raft_consensus.cc:729] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a5c1e9863e481387554126e1f6383a, State: Initialized, Role: FOLLOWER
I20260926 06:55:54.543480 16756 consensus_queue.cc:260] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:54.543782 16756 ts_tablet_manager.cc:1428] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:55:54.547128 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.533605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:58915 (local address 127.0.191.254:36031)
0926 06:55:54.533789 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:54.533798 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:54.534118 (+ 320us) server_negotiation.cc:407] Connection header received
0926 06:55:54.534804 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:54.534812 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:54.534874 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:54.534982 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:54.536008 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.537144 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.538658 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.539360 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.539448 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.542517 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:54.542521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:54.542521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:54.542534 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:54.544823 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:54.545351 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:54.545355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:54.545355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:54.545509 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:54.545847 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:54.545849 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:54.545849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:54.546173 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:54.546236 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:54.546680 (+ 444us) server_negotiation.cc:299] Negotiation successful
0926 06:55:54.546845 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260926 06:55:54.547782 16867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.533820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:58915)
0926 06:55:54.533984 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:54.534022 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:54.534552 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:54.535224 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:54.535228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:54.535614 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:54.535878 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.535884 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.537270 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.537271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.538556 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.538559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.539595 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.539596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.539775 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.540360 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:54.540371 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:54.542386 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:54.544925 (+ 2539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:54.544930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:54.544932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:54.545244 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:54.545600 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:54.545601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:54.545601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:54.545754 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:54.546293 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:54.546300 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:54.546462 (+ 162us) client_negotiation.cc:769] Sending connection context
0926 06:55:54.547326 (+ 864us) client_negotiation.cc:240] Negotiation successful
0926 06:55:54.547519 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":103,"threads_started":1}
I20260926 06:55:54.548588 16864 heartbeater.cc:347] Connected to a master server at 127.0.191.254:36031
I20260926 06:55:54.548704 16864 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:54.548873 16864 heartbeater.cc:511] Master 127.0.191.254:36031 requested a full tablet report, sending...
I20260926 06:55:54.549654 16676 ts_manager.cc:194] Registered new tserver with Master: 04a5c1e9863e481387554126e1f6383a (127.0.191.193:33491)
I20260926 06:55:54.550757 16676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:58915
I20260926 06:55:54.551652 16864 heartbeater.cc:503] Master 127.0.191.254:36031 was elected leader, sending a full tablet report...
I20260926 06:55:54.556391 16871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.550227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:33491 (local address 127.0.0.1:44910)
0926 06:55:54.552040 (+ 1813us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:54.552075 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:54.552199 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:54.552405 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:54.552409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:54.552411 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:54.552501 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.552502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.553803 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.553804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.555109 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.555112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.556107 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.556108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.556197 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.556209 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 06:55:54.556268 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 06:55:54.556314 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1624,"thread_start_us":92,"threads_started":1}
I20260926 06:55:54.556723 16872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.551028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44910 (local address 127.0.191.193:33491)
0926 06:55:54.552022 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:54.552026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:54.552132 (+ 106us) server_negotiation.cc:407] Connection header received
0926 06:55:54.552289 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:54.552291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:54.552306 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:54.552383 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:54.552578 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.553703 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.555201 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.555937 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.556003 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.556069 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:54.556617 (+ 548us) server_negotiation.cc:299] Negotiation successful
0926 06:55:54.556656 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260926 06:55:54.557337 16785 tablet_service.cc:1514] Processing DeleteTablet for tablet b7bcef7a74a5426683843a0aaf39c1bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:34 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:44910
I20260926 06:55:54.558015 16873 tablet_replica.cc:330] stopping tablet replica
I20260926 06:55:54.558122 16873 raft_consensus.cc:2229] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:55:54.558192 16873 raft_consensus.cc:2258] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:55:54.558496 16873 ts_tablet_manager.cc:1905] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:55:54.560949 16873 ts_tablet_manager.cc:1918] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:55:54.561019 16873 log.cc:1196] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal/wals/b7bcef7a74a5426683843a0aaf39c1bb
I20260926 06:55:54.561244 16873 ts_tablet_manager.cc:1939] T b7bcef7a74a5426683843a0aaf39c1bb P 04a5c1e9863e481387554126e1f6383a: Deleting consensus metadata
I20260926 06:55:54.561805 16660 catalog_manager.cc:4841] TS 04a5c1e9863e481387554126e1f6383a (127.0.191.193:33491): tablet b7bcef7a74a5426683843a0aaf39c1bb (table test-table [id=adae5a31ea554cc292d031c9e0424e27]) successfully deleted
W20260926 06:55:54.622822 16868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:54.623039 16868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:54.623075 16868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:54.624215 16868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:54.624290 16868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:54.624338 16868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:55:54.625905 16868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:42177
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=39213
--tserver_master_addrs=127.0.191.254:36031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:54.626163 16868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:54.626418 16868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:54.631407 16879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:54.631428 16881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:54.631456 16878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:54.631623 16868 server_base.cc:1034] running on GCE node
I20260926 06:55:54.631822 16868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:54.632032 16868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:54.633179 16868 hybrid_clock.cc:648] HybridClock initialized: now 1790405754633155 us; error 37 us; skew 500 ppm
Sep 26 06:55:54 dist-test-slave-ws74 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405754, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:54.635675 16868 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:55:54.636132 16868 webserver.cc:460] Webserver started at http://127.0.191.194:39213/ using document root <none> and password file <none>
I20260926 06:55:54.636304 16868 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:54.636348 16868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:54.637637 16868 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:55:54.638201 16888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:54.638391 16868 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:55:54.638477 16868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "00a0eeeabef14875aee59035e4cf1526"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "c70632b8614752ab7f77c895ff4dc648"
server_key_iv: "5fc9e2b7616dbb927bfc15c121ed843c"
server_key_version: "encryptionkey@0"
I20260926 06:55:54.638784 16868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:54.686427 16868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:54.688450 16868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:54.688623 16868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:54.688867 16868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:54.689177 16868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:54.689239 16868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:54.689275 16868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:54.689304 16868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:54.695106 16868 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:42177
I20260926 06:55:54.695154 17001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:42177 every 8 connection(s)
I20260926 06:55:54.695472 16868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:55:54.701051 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 16868
I20260926 06:55:54.701388 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:40299
--local_ip_for_outbound_sockets=127.0.191.195
--tserver_master_addrs=127.0.191.254:36031
--webserver_port=42901
--webserver_interface=127.0.191.195
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 06:55:54 dist-test-slave-ws74 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405754, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:54.705228 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.696178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:56927 (local address 127.0.191.254:36031)
0926 06:55:54.696221 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:54.696223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:54.696441 (+ 218us) server_negotiation.cc:407] Connection header received
0926 06:55:54.696969 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:54.696971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:54.696986 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:54.697038 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:54.698015 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.698518 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.699499 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.699710 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.699796 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.702086 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:54.702088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:54.702088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:54.702095 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:54.703813 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:54.704208 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:54.704209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:54.704210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:54.704273 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:54.704505 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:54.704506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:54.704506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:54.704653 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:54.704725 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:54.704975 (+ 250us) server_negotiation.cc:299] Negotiation successful
0926 06:55:54.705075 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:55:54.705299 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.696154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:56927)
0926 06:55:54.696317 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:54.696352 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:54.696853 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:54.697104 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:54.697108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:54.697726 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:54.697914 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.697919 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.698614 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.698615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.699408 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.699410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.699806 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.699807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.699932 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.700374 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:54.700382 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:54.701977 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:54.703913 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:54.703918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:54.703920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:54.704133 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:54.704342 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:54.704343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:54.704343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:54.704439 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:54.704717 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:54.704721 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:54.704830 (+ 109us) client_negotiation.cc:769] Sending connection context
0926 06:55:54.704948 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 06:55:54.705103 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260926 06:55:54.705806 17002 heartbeater.cc:347] Connected to a master server at 127.0.191.254:36031
I20260926 06:55:54.705886 17002 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:54.706048 17002 heartbeater.cc:511] Master 127.0.191.254:36031 requested a full tablet report, sending...
I20260926 06:55:54.706440 16676 ts_manager.cc:194] Registered new tserver with Master: 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177)
I20260926 06:55:54.706809 16676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:56927
W20260926 06:55:54.782038 17005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:55:54.782260 17005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:55:54.782295 17005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:55:54.783440 17005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:55:54.783504 17005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:55:54.783546 17005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:55:54.784940 17005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:40299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=42901
--tserver_master_addrs=127.0.191.254:36031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:55:54.785187 17005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:55:54.785432 17005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:55:54.790283 17013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:55:54.790346 17011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:54.790490 17005 server_base.cc:1034] running on GCE node
W20260926 06:55:54.790560 17010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:55:54.790805 17005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:55:54.791067 17005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:55:54.792227 17005 hybrid_clock.cc:648] HybridClock initialized: now 1790405754792208 us; error 31 us; skew 500 ppm
Sep 26 06:55:54 dist-test-slave-ws74 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405754, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:55:54.794716 17005 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:55:54.795186 17005 webserver.cc:460] Webserver started at http://127.0.191.195:42901/ using document root <none> and password file <none>
I20260926 06:55:54.795415 17005 fs_manager.cc:359] Metadata directory not provided
I20260926 06:55:54.795496 17005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:55:54.796797 17005 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:55:54.797518 17020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:54.797703 17005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:55:54.797788 17005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "460f16e281384d10aa8a29107c8665a0"
format_stamp: "Formatted at 2026-09-26 06:55:34 on dist-test-slave-ws74"
server_key: "542857a999462d40e5a592f1bf2f3f3f"
server_key_iv: "b588878465ed31d923872d6ed1943e58"
server_key_version: "encryptionkey@0"
I20260926 06:55:54.798123 17005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:55:54.818208 17005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:55:54.820014 17005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:55:54.820173 17005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:55:54.820399 17005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:55:54.820703 17005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:55:54.820755 17005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:54.820796 17005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:55:54.820822 17005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:55:54.826622 17005 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:40299
I20260926 06:55:54.826664 17133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:40299 every 8 connection(s)
I20260926 06:55:54.827042 17005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:55:54 dist-test-slave-ws74 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405754, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:55:54.835829 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.827698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:39459 (local address 127.0.191.254:36031)
0926 06:55:54.827742 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:54.827744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:54.827971 (+ 227us) server_negotiation.cc:407] Connection header received
0926 06:55:54.828495 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:54.828496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:54.828510 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:54.828561 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:54.829536 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.830027 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.831002 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.831214 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.831298 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.833261 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:54.833263 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:54.833263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:54.833270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:54.834705 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:54.835069 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:54.835071 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:54.835071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:54.835128 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:54.835338 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:54.835338 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:54.835339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:54.835457 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:54.835508 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:54.835706 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 06:55:54.835767 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:55:54.836037 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.827694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:39459)
0926 06:55:54.827853 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:54.827892 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:54.828384 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:54.828627 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:54.828631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:54.829194 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:55:54.829428 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.829434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.830116 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.830117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.830911 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.830913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.831306 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.831307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.831431 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.831841 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:55:54.831849 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:55:54.833171 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:55:54.834773 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:54.834778 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:54.834779 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:54.834977 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:54.835193 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:55:54.835193 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:55:54.835194 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:55:54.835282 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:55:54.835510 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:55:54.835514 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:55:54.835617 (+ 103us) client_negotiation.cc:769] Sending connection context
0926 06:55:54.835694 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 06:55:54.835800 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260926 06:55:54.836504 17134 heartbeater.cc:347] Connected to a master server at 127.0.191.254:36031
I20260926 06:55:54.836620 17134 heartbeater.cc:464] Registering TS with master...
I20260926 06:55:54.836805 17134 heartbeater.cc:511] Master 127.0.191.254:36031 requested a full tablet report, sending...
I20260926 06:55:54.836951 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 17005
I20260926 06:55:54.837239 16676 ts_manager.cc:194] Registered new tserver with Master: 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299)
I20260926 06:55:54.837767 16676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:39459
I20260926 06:55:54.842736 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.837321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.0.191.254:36031)
0926 06:55:54.837418 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:54.837420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:54.837553 (+ 133us) server_negotiation.cc:407] Connection header received
0926 06:55:54.837606 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:54.837607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:54.837615 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:54.837654 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:55:54.838391 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.838813 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.839500 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.839665 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.839712 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.840712 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:55:54.840713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:55:54.840713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:55:54.840718 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:55:54.841836 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:54.842139 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:54.842140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:54.842140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:54.842185 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:55:54.842318 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:55:54.842319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:55:54.842319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:55:54.842408 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:55:54.842545 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:54.842627 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 06:55:54.842681 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 06:55:54.843412 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:55:54.845719 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.843710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35684 (local address 127.0.191.254:36031)
0926 06:55:54.843747 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:54.843749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:54.843823 (+ 74us) server_negotiation.cc:407] Connection header received
0926 06:55:54.843888 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:54.843888 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:54.843912 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:54.843968 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:55:54.844034 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.844450 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.845103 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.845263 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.845308 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.845468 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:55:54.845555 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:55:54.845613 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:54.845626 (+ 13us) server_negotiation.cc:299] Negotiation successful
0926 06:55:54.845639 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:55:54.845983 16676 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:35684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:55:54.847469 16676 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:55:54.849545 16785 tablet_service.cc:1467] Processing CreateTablet for tablet a747382066e24fa9924e168a652eec29 (DEFAULT_TABLE table=test-table [id=abe4928630f34e5082c14a11a24b7858]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:54.849759 16785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a747382066e24fa9924e168a652eec29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:54.851459 16756 tablet_bootstrap.cc:492] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a: Bootstrap starting.
I20260926 06:55:54.851836 16756 tablet_bootstrap.cc:654] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:54.852387 16756 tablet_bootstrap.cc:492] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a: No bootstrap required, opened a new log
I20260926 06:55:54.852453 16756 ts_tablet_manager.cc:1397] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 06:55:54.852602 16756 raft_consensus.cc:348] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:54.852676 16756 raft_consensus.cc:374] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:54.852725 16756 raft_consensus.cc:729] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a5c1e9863e481387554126e1f6383a, State: Initialized, Role: FOLLOWER
I20260926 06:55:54.852768 16756 consensus_queue.cc:260] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:54.852862 16756 ts_tablet_manager.cc:1428] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 06:55:54.853178 16871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.849188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:40299 (local address 127.0.0.1:38754)
0926 06:55:54.849249 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:54.849259 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:54.849293 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:54.849713 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:54.849714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:54.849716 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:54.849762 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.849763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.850931 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.850932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.851869 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.851871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.852988 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.852989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.853047 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.853050 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:55:54.853079 (+ 29us) client_negotiation.cc:240] Negotiation successful
0926 06:55:54.853115 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 06:55:54.853618 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.849188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:42177 (local address 127.0.0.1:40548)
0926 06:55:54.849527 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:54.849537 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:54.849590 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:54.850086 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:54.850087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:54.850090 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:54.850127 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.850129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.851375 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.851376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.852414 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.852416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.853347 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.853348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.853445 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.853449 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:55:54.853499 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 06:55:54.853538 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":35,"thread_start_us":46,"threads_started":1}
I20260926 06:55:54.853653 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.849324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.0.191.195:40299)
0926 06:55:54.849524 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:54.849528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:54.849538 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:55:54.849571 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:54.849574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:54.849589 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:54.849717 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:54.849839 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.850840 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.851956 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.852895 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.853478 (+ 583us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.853537 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:54.853563 (+ 26us) server_negotiation.cc:299] Negotiation successful
0926 06:55:54.853598 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":133,"threads_started":1}
I20260926 06:55:54.853866 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.849358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40548 (local address 127.0.191.194:42177)
0926 06:55:54.849922 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:54.849926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:54.849936 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:55:54.849975 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:54.849977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:54.849993 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:54.850076 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:54.850189 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.851261 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.852500 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.853205 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.853274 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.853332 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:54.853735 (+ 403us) server_negotiation.cc:299] Negotiation successful
0926 06:55:54.853792 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":102,"threads_started":1}
I20260926 06:55:54.854223 17068 tablet_service.cc:1467] Processing CreateTablet for tablet a747382066e24fa9924e168a652eec29 (DEFAULT_TABLE table=test-table [id=abe4928630f34e5082c14a11a24b7858]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:54.854571 17068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a747382066e24fa9924e168a652eec29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:54.857023 17143 tablet_bootstrap.cc:492] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0: Bootstrap starting.
I20260926 06:55:54.857604 16936 tablet_service.cc:1467] Processing CreateTablet for tablet a747382066e24fa9924e168a652eec29 (DEFAULT_TABLE table=test-table [id=abe4928630f34e5082c14a11a24b7858]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:55:54.857728 17143 tablet_bootstrap.cc:654] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:54.857903 16936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a747382066e24fa9924e168a652eec29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:55:54.858255 17143 log.cc:824] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:54.858958 17143 tablet_bootstrap.cc:492] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0: No bootstrap required, opened a new log
I20260926 06:55:54.859009 17143 ts_tablet_manager.cc:1397] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:55:54.860512 17143 raft_consensus.cc:348] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:54.860668 17143 raft_consensus.cc:374] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:54.860709 17143 raft_consensus.cc:729] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460f16e281384d10aa8a29107c8665a0, State: Initialized, Role: FOLLOWER
I20260926 06:55:54.860754 17145 tablet_bootstrap.cc:492] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526: Bootstrap starting.
I20260926 06:55:54.860841 17143 consensus_queue.cc:260] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:54.861135 17143 ts_tablet_manager.cc:1428] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:55:54.861166 17134 heartbeater.cc:503] Master 127.0.191.254:36031 was elected leader, sending a full tablet report...
I20260926 06:55:54.861433 17145 tablet_bootstrap.cc:654] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526: Neither blocks nor log segments found. Creating new log.
I20260926 06:55:54.861979 17145 log.cc:824] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526: Log is configured to *not* fsync() on all Append() calls
I20260926 06:55:54.862651 17145 tablet_bootstrap.cc:492] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526: No bootstrap required, opened a new log
I20260926 06:55:54.862733 17145 ts_tablet_manager.cc:1397] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:55:54.864198 17145 raft_consensus.cc:348] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:54.864349 17145 raft_consensus.cc:374] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:55:54.864395 17145 raft_consensus.cc:729] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a0eeeabef14875aee59035e4cf1526, State: Initialized, Role: FOLLOWER
I20260926 06:55:54.864511 17145 consensus_queue.cc:260] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:54.864787 17145 ts_tablet_manager.cc:1428] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:55:54.864799 17002 heartbeater.cc:503] Master 127.0.191.254:36031 was elected leader, sending a full tablet report...
I20260926 06:55:54.895262 16870 raft_consensus.cc:482] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:55:54.895409 16870 raft_consensus.cc:504] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:54.895686 16870 leader_election.cc:288] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299), 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177)
I20260926 06:55:54.899187 16867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.895857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:40299 (local address 127.0.191.193:46775)
0926 06:55:54.895904 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:54.895918 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:54.895969 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:54.896218 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:54.896219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:54.896221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:54.896306 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.896307 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.897071 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.897072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.898374 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.898376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.898993 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.898994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.899051 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.899054 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:55:54.899095 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 06:55:54.899124 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260926 06:55:54.899235 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.896046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:46775 (local address 127.0.191.195:40299)
0926 06:55:54.896096 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:54.896098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:54.896106 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:55:54.896134 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:54.896135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:54.896144 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:54.896207 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:54.896393 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.896978 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.898457 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.898913 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.898986 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.898995 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:54.899117 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 06:55:54.899161 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:55:54.899628 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.895941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:46937 (local address 127.0.191.194:42177)
0926 06:55:54.896014 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:54.896017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:54.896259 (+ 242us) server_negotiation.cc:407] Connection header received
0926 06:55:54.896328 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:54.896329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:54.896336 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:54.896405 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:55:54.896810 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.897536 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.898710 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.899278 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.899345 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.899352 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:54.899527 (+ 175us) server_negotiation.cc:299] Negotiation successful
0926 06:55:54.899562 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 06:55:54.899574 17088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a747382066e24fa9924e168a652eec29" candidate_uuid: "04a5c1e9863e481387554126e1f6383a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460f16e281384d10aa8a29107c8665a0" is_pre_election: true
I20260926 06:55:54.899657 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.895857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:42177 (local address 127.0.191.193:46937)
0926 06:55:54.896201 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:54.896211 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:55:54.896271 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:55:54.896426 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:55:54.896426 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:55:54.896428 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:55:54.896730 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.896731 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.897642 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.897643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.898640 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:55:54.898642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.899394 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.899395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:55:54.899452 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.899454 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:55:54.899523 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 06:55:54.899560 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":133,"thread_start_us":116,"threads_started":1}
I20260926 06:55:54.899780 17088 raft_consensus.cc:2454] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04a5c1e9863e481387554126e1f6383a in term 0.
I20260926 06:55:54.900009 16751 leader_election.cc:302] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a5c1e9863e481387554126e1f6383a, 460f16e281384d10aa8a29107c8665a0; no voters:
I20260926 06:55:54.900013 16956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a747382066e24fa9924e168a652eec29" candidate_uuid: "04a5c1e9863e481387554126e1f6383a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a0eeeabef14875aee59035e4cf1526" is_pre_election: true
I20260926 06:55:54.900168 16956 raft_consensus.cc:2454] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04a5c1e9863e481387554126e1f6383a in term 0.
I20260926 06:55:54.900151 16870 raft_consensus.cc:2781] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:55:54.900221 16870 raft_consensus.cc:482] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:55:54.900247 16870 raft_consensus.cc:3037] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:54.901022 16870 raft_consensus.cc:504] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:54.901167 16870 leader_election.cc:288] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 1 election: Requested vote from peers 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299), 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177)
I20260926 06:55:54.901298 16956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a747382066e24fa9924e168a652eec29" candidate_uuid: "04a5c1e9863e481387554126e1f6383a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a0eeeabef14875aee59035e4cf1526"
I20260926 06:55:54.901298 17088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a747382066e24fa9924e168a652eec29" candidate_uuid: "04a5c1e9863e481387554126e1f6383a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460f16e281384d10aa8a29107c8665a0"
I20260926 06:55:54.901414 17088 raft_consensus.cc:3037] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:54.901400 16956 raft_consensus.cc:3037] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:55:54.902253 17088 raft_consensus.cc:2454] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04a5c1e9863e481387554126e1f6383a in term 1.
I20260926 06:55:54.902253 16956 raft_consensus.cc:2454] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04a5c1e9863e481387554126e1f6383a in term 1.
I20260926 06:55:54.902458 16751 leader_election.cc:302] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a5c1e9863e481387554126e1f6383a, 460f16e281384d10aa8a29107c8665a0; no voters:
I20260926 06:55:54.902573 16870 raft_consensus.cc:2781] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:55:54.902746 16870 raft_consensus.cc:686] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 LEADER]: Becoming Leader. State: Replica: 04a5c1e9863e481387554126e1f6383a, State: Running, Role: LEADER
I20260926 06:55:54.902833 16870 consensus_queue.cc:237] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:55:54.903607 16676 catalog_manager.cc:5495] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a reported cstate change: term changed from 0 to 1, leader changed from <none> to 04a5c1e9863e481387554126e1f6383a (127.0.191.193). New cstate: current_term: 1 leader_uuid: "04a5c1e9863e481387554126e1f6383a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } health_report { overall_health: HEALTHY } } }
I20260926 06:55:54.913403 16872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.911042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44914 (local address 127.0.191.193:33491)
0926 06:55:54.911085 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:55:54.911087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:55:54.911097 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:55:54.911179 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:55:54.911180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:55:54.911197 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:55:54.911265 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:55:54.911363 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.911877 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.912626 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:55:54.912819 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:55:54.912908 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:55:54.913075 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:55:54.913169 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:55:54.913243 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:55:54.913288 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 06:55:54.913308 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:55:54.914726 16956 raft_consensus.cc:1261] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526 [term 1 FOLLOWER]: Refusing update from remote peer 04a5c1e9863e481387554126e1f6383a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:55:54.914747 17088 raft_consensus.cc:1261] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0 [term 1 FOLLOWER]: Refusing update from remote peer 04a5c1e9863e481387554126e1f6383a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:55:54.915036 16870 consensus_queue.cc:1035] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [LEADER]: Connected to new peer: Peer: permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:55:54.915166 17150 consensus_queue.cc:1035] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [LEADER]: Connected to new peer: Peer: permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:55:54.917606 17152 mvcc.cc:205] Tried to move back new op lower bound from 7333501972128460800 to 7333501972082565120. Current Snapshot: MvccSnapshot[applied={T|T < 7333501972128460800}]
I20260926 06:55:54.917865 17154 mvcc.cc:205] Tried to move back new op lower bound from 7333501972128460800 to 7333501972082565120. Current Snapshot: MvccSnapshot[applied={T|T < 7333501972128460800}]
I20260926 06:55:54.920042 16676 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:35684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:55:54.920140 16676 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:35684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:55:54.920868 16676 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Sending DeleteTablet for 3 replicas of tablet a747382066e24fa9924e168a652eec29
I20260926 06:55:54.921167 16785 tablet_service.cc:1514] Processing DeleteTablet for tablet a747382066e24fa9924e168a652eec29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:54 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:44910
I20260926 06:55:54.921271 16873 tablet_replica.cc:330] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a: stopping tablet replica
I20260926 06:55:54.921221 17068 tablet_service.cc:1514] Processing DeleteTablet for tablet a747382066e24fa9924e168a652eec29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:54 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:38754
I20260926 06:55:54.921301 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 16714
I20260926 06:55:54.921326 16873 raft_consensus.cc:2229] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:55:54.921433 16873 raft_consensus.cc:2258] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:55:54.922107 16936 tablet_service.cc:1514] Processing DeleteTablet for tablet a747382066e24fa9924e168a652eec29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:55:54 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:40548
I20260926 06:55:54.922324 17163 tablet_replica.cc:330] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526: stopping tablet replica
I20260926 06:55:54.922425 17162 tablet_replica.cc:330] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0: stopping tablet replica
I20260926 06:55:54.922428 17163 raft_consensus.cc:2229] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:55:54.922509 17162 raft_consensus.cc:2229] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:55:54.922534 17163 raft_consensus.cc:2258] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:55:54.922595 17162 raft_consensus.cc:2258] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:55:54.922976 17163 ts_tablet_manager.cc:1905] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:55:54.922987 17162 ts_tablet_manager.cc:1905] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:55:54.924968 17163 ts_tablet_manager.cc:1918] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:55:54.924968 17162 ts_tablet_manager.cc:1918] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:55:54.925046 17163 log.cc:1196] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/a747382066e24fa9924e168a652eec29
I20260926 06:55:54.925046 17162 log.cc:1196] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/a747382066e24fa9924e168a652eec29
I20260926 06:55:54.925364 17162 ts_tablet_manager.cc:1939] T a747382066e24fa9924e168a652eec29 P 460f16e281384d10aa8a29107c8665a0: Deleting consensus metadata
I20260926 06:55:54.925455 17163 ts_tablet_manager.cc:1939] T a747382066e24fa9924e168a652eec29 P 00a0eeeabef14875aee59035e4cf1526: Deleting consensus metadata
I20260926 06:55:54.925839 16661 catalog_manager.cc:4841] TS 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299): tablet a747382066e24fa9924e168a652eec29 (table test-table [id=abe4928630f34e5082c14a11a24b7858]) successfully deleted
I20260926 06:55:54.925947 16662 catalog_manager.cc:4841] TS 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177): tablet a747382066e24fa9924e168a652eec29 (table test-table [id=abe4928630f34e5082c14a11a24b7858]) successfully deleted
W20260926 06:55:54.927157 16660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:33491 (error 108)
I20260926 06:55:54.927413 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 16868
I20260926 06:55:54.927610 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:55:54.927499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:33491 (local address 127.0.0.1:44920)
0926 06:55:54.927546 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 06:55:54.927576 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:33491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 06:55:54.927707 16660 catalog_manager.cc:4584] TS 04a5c1e9863e481387554126e1f6383a (127.0.191.193:33491): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a747382066e24fa9924e168a652eec29: Network error: Client connection negotiation failed: client connection to 127.0.191.193:33491: connect: Connection refused (error 111)
I20260926 06:55:54.932425 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 17005
I20260926 06:55:54.936976 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 16643
I20260926 06:56:14.942382 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:36031
--webserver_interface=127.0.191.254
--webserver_port=39555
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:36031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:15.022248 17165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:15.022451 17165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:15.022473 17165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:15.023576 17165 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 06:56:15.023610 17165 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 06:56:15.023623 17165 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 06:56:15.023634 17165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:15.023648 17165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:15.023660 17165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:15.023672 17165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:15.023684 17165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:15.025071 17165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:36031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:36031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=39555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:15.025255 17165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:15.025496 17165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:15.030829 17170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:15.030834 17171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:15.030845 17173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:15.030893 17165 server_base.cc:1034] running on GCE node
I20260926 06:56:15.031337 17165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:15.031623 17165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:15.032766 17165 hybrid_clock.cc:648] HybridClock initialized: now 1790405775032756 us; error 31 us; skew 500 ppm
Sep 26 06:56:15 dist-test-slave-ws74 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:15.038709 17165 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:56:15.039192 17165 webserver.cc:460] Webserver started at http://127.0.191.254:39555/ using document root <none> and password file <none>
I20260926 06:56:15.039479 17165 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:15.039552 17165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:15.040959 17165 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:56:15.041831 17180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:15.042119 17165 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:56:15.042236 17165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "f17c7b1a08bc494dbea725394e0f83c4"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "216bf1c1a87fd4e955d769bd8b69af03"
server_key_iv: "a8a6d81c432f0951f6d364a250aca844"
server_key_version: "encryptionkey@0"
I20260926 06:56:15.042606 17165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:15.050386 17165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:15.052407 17165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:15.052582 17165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:15.056944 17165 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:36031
I20260926 06:56:15.057024 17232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:36031 every 8 connection(s)
I20260926 06:56:15.057526 17165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:15.059216 17233 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:56:15.059897 17233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Bootstrap starting.
I20260926 06:56:15.067549 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 17165
I20260926 06:56:15.067916 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:33491
--local_ip_for_outbound_sockets=127.0.191.193
--tserver_master_addrs=127.0.191.254:36031
--webserver_port=33807
--webserver_interface=127.0.191.193
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:56:15.069912 17233 log.cc:824] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:15.073354 17233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:56:15.073752 17233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Bootstrap complete.
I20260926 06:56:15.077824 17233 raft_consensus.cc:348] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } }
I20260926 06:56:15.078229 17233 raft_consensus.cc:729] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f17c7b1a08bc494dbea725394e0f83c4, State: Initialized, Role: FOLLOWER
I20260926 06:56:15.078425 17233 consensus_queue.cc:260] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } }
I20260926 06:56:15.078518 17233 raft_consensus.cc:388] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:15.078583 17233 raft_consensus.cc:482] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:15.078641 17233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 2 FOLLOWER]: Advancing to term 3
I20260926 06:56:15.080243 17233 raft_consensus.cc:504] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } }
I20260926 06:56:15.080356 17233 leader_election.cc:302] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f17c7b1a08bc494dbea725394e0f83c4; no voters:
I20260926 06:56:15.080520 17233 leader_election.cc:288] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 06:56:15.080612 17237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 06:56:15.080765 17233 sys_catalog.cc:564] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:15.080799 17237 raft_consensus.cc:686] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [term 3 LEADER]: Becoming Leader. State: Replica: f17c7b1a08bc494dbea725394e0f83c4, State: Running, Role: LEADER
I20260926 06:56:15.080907 17237 consensus_queue.cc:237] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } }
I20260926 06:56:15.081171 17238 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f17c7b1a08bc494dbea725394e0f83c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } } }
I20260926 06:56:15.081251 17238 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:15.081457 17244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:15.081565 17239 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f17c7b1a08bc494dbea725394e0f83c4. Latest consensus state: current_term: 3 leader_uuid: "f17c7b1a08bc494dbea725394e0f83c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17c7b1a08bc494dbea725394e0f83c4" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36031 } } }
I20260926 06:56:15.081686 17239 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:15.082340 17244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:15.082512 17244 catalog_manager.cc:1259] Loaded cluster ID: ae3a63648228414ca0f2752415eb63eb
I20260926 06:56:15.082690 17244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:15.084012 17244 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:15.084189 17244 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Loaded TSK: 0
I20260926 06:56:15.084241 17244 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f17c7b1a08bc494dbea725394e0f83c4: Loaded TSK: 1
F20260926 06:56:15.084488 17244 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7c341579ad google::LogMessage::Fail() at ??:0
@ 0x7f7c3415a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f7c341574de google::LogMessage::Flush() at ??:0
@ 0x7f7c34158a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7c368c4104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f7c368c1d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f7c3c0b11cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f7c3c0b148e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f7c3c0b1594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f7c3c077c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f7c3c081ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7c360250d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7c3601a954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7c36e7e6db start_thread at ??:0
@ 0x7f7c32c5561f clone at ??:0
W20260926 06:56:15.171552 17235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:15.171743 17235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:15.171765 17235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:15.172912 17235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:15.172953 17235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:15.172979 17235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:56:15.174475 17235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:33491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=33807
--tserver_master_addrs=127.0.191.254:36031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:15.174707 17235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:15.174943 17235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:15.179915 17277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:15.179931 17274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:15.179931 17275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:15.180253 17235 server_base.cc:1034] running on GCE node
I20260926 06:56:15.180508 17235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:15.180768 17235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:15.181963 17235 hybrid_clock.cc:648] HybridClock initialized: now 1790405775181938 us; error 40 us; skew 500 ppm
Sep 26 06:56:15 dist-test-slave-ws74 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:15.184484 17235 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:56:15.184986 17235 webserver.cc:460] Webserver started at http://127.0.191.193:33807/ using document root <none> and password file <none>
I20260926 06:56:15.185227 17235 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:15.185271 17235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:15.186617 17235 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:15.187301 17284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:15.187491 17235 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:15.187600 17235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "04a5c1e9863e481387554126e1f6383a"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "ce83d71e0949faf8d82ad6a4f20d4e21"
server_key_iv: "d68c434d4cd123eadac903539e149abd"
server_key_version: "encryptionkey@0"
I20260926 06:56:15.188082 17235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:15.204156 17235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:15.207034 17235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:15.207217 17235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:15.207509 17235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:15.208068 17291 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 06:56:15.209636 17235 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 06:56:15.209699 17235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 06:56:15.209738 17235 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 06:56:15.210706 17235 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 06:56:15.210758 17235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 06:56:15.210821 17291 tablet_bootstrap.cc:492] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a: Bootstrap starting.
I20260926 06:56:15.218575 17235 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:33491
I20260926 06:56:15.218789 17398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:33491 every 8 connection(s)
I20260926 06:56:15.219064 17235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:56:15.221172 17291 log.cc:824] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:15.223258 17291 tablet_bootstrap.cc:492] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:56:15.223547 17291 tablet_bootstrap.cc:492] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a: Bootstrap complete.
I20260926 06:56:15.224416 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 17235
I20260926 06:56:15.224628 17291 ts_tablet_manager.cc:1397] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260926 06:56:15.224716 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:42177
--local_ip_for_outbound_sockets=127.0.191.194
--tserver_master_addrs=127.0.191.254:36031
--webserver_port=39213
--webserver_interface=127.0.191.194
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:56:15.226367 17291 raft_consensus.cc:348] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:56:15.226600 17291 raft_consensus.cc:729] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a5c1e9863e481387554126e1f6383a, State: Initialized, Role: FOLLOWER
I20260926 06:56:15.226776 17291 consensus_queue.cc:260] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:56:15.227063 17291 ts_tablet_manager.cc:1428] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a: Time spent starting tablet: real 0.002s user 0.004s sys 0.002s
W20260926 06:56:15.298926 17401 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:56:15.219805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:40689)
0926 06:56:15.220033 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:15.220092 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:15.220857 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:15.298720 (+ 77863us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":110,"threads_started":1}
W20260926 06:56:15.299237 17399 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:36031 (0 consecutive failures): Network error: Failed to ping master at 127.0.191.254:36031: Client connection negotiation failed: client connection to 127.0.191.254:36031: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 06:56:15.299567 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:15.299449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:48535)
0926 06:56:15.299503 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:15.299539 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 06:56:15.299921 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:15.299835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:40727)
0926 06:56:15.299880 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:15.299902 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 06:56:15.300038 17399 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 06:56:15.310318 17403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:15.310544 17403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:15.310583 17403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:15.311739 17403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:15.311803 17403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:15.311848 17403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:56:15.313298 17403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:42177
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=39213
--tserver_master_addrs=127.0.191.254:36031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:15.313562 17403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:15.313740 17403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:56:15.318581 17403 server_base.cc:1034] running on GCE node
W20260926 06:56:15.318543 17412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:15.318553 17409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:15.318596 17410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:15.318928 17403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:15.319169 17403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:15.320325 17403 hybrid_clock.cc:648] HybridClock initialized: now 1790405775320304 us; error 35 us; skew 500 ppm
Sep 26 06:56:15 dist-test-slave-ws74 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:15.322755 17403 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:56:15.323164 17403 webserver.cc:460] Webserver started at http://127.0.191.194:39213/ using document root <none> and password file <none>
I20260926 06:56:15.323395 17403 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:15.323465 17403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:15.324729 17403 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:56:15.325315 17419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:15.325520 17403 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:56:15.325614 17403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "00a0eeeabef14875aee59035e4cf1526"
format_stamp: "Formatted at 2026-09-26 06:55:33 on dist-test-slave-ws74"
server_key: "c70632b8614752ab7f77c895ff4dc648"
server_key_iv: "5fc9e2b7616dbb927bfc15c121ed843c"
server_key_version: "encryptionkey@0"
I20260926 06:56:15.325939 17403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:15.346180 17403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:15.347954 17403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:15.348133 17403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:15.348372 17403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:15.348685 17403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:15.348739 17403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:15.348779 17403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:15.348806 17403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:15.354678 17403 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:42177
I20260926 06:56:15.354717 17532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:42177 every 8 connection(s)
I20260926 06:56:15.355099 17403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:56:15.356139 17535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:15.355782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:42561)
0926 06:56:15.355932 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:15.356003 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
W20260926 06:56:15.356549 17533 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:36031 (0 consecutive failures): Network error: Failed to ping master at 127.0.191.254:36031: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
I20260926 06:56:15.356805 17535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:15.356716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:59717)
0926 06:56:15.356765 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:15.356789 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 06:56:15.357087 17535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:15.357015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:57995)
0926 06:56:15.357054 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:15.357074 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 06:56:15.357201 17533 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 06:56:15.360484 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 17403
I20260926 06:56:15.360759 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:40299
--local_ip_for_outbound_sockets=127.0.191.195
--tserver_master_addrs=127.0.191.254:36031
--webserver_port=42901
--webserver_interface=127.0.191.195
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:15.440495 17536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:15.440709 17536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:15.440735 17536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:15.441915 17536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:15.441960 17536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:15.441989 17536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:56:15.443444 17536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:40299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=42901
--tserver_master_addrs=127.0.191.254:36031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:15.443671 17536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:15.443912 17536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:15.448989 17544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:15.449030 17542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:15.449028 17536 server_base.cc:1034] running on GCE node
W20260926 06:56:15.449190 17541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:15.449476 17536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:15.449730 17536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:15.450888 17536 hybrid_clock.cc:648] HybridClock initialized: now 1790405775450878 us; error 37 us; skew 500 ppm
Sep 26 06:56:15 dist-test-slave-ws74 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:15.453358 17536 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:56:15.453846 17536 webserver.cc:460] Webserver started at http://127.0.191.195:42901/ using document root <none> and password file <none>
I20260926 06:56:15.454095 17536 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:15.454164 17536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:15.455528 17536 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:56:15.456099 17551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:15.456321 17536 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:56:15.456436 17536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "460f16e281384d10aa8a29107c8665a0"
format_stamp: "Formatted at 2026-09-26 06:55:34 on dist-test-slave-ws74"
server_key: "542857a999462d40e5a592f1bf2f3f3f"
server_key_iv: "b588878465ed31d923872d6ed1943e58"
server_key_version: "encryptionkey@0"
I20260926 06:56:15.456777 17536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:15.474913 17536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:15.476938 17536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:15.477124 17536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:15.477386 17536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:15.477710 17536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:15.477763 17536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:15.477807 17536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:15.477835 17536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:15.483494 17536 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:40299
I20260926 06:56:15.483534 17664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:40299 every 8 connection(s)
I20260926 06:56:15.483920 17536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:56:15.485059 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:15.484606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:45855)
0926 06:56:15.484788 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:15.484897 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":104,"threads_started":1}
I20260926 06:56:15.485466 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 17536
W20260926 06:56:15.485586 17665 heartbeater.cc:644] Failed to heartbeat to 127.0.191.254:36031 (0 consecutive failures): Network error: Failed to ping master at 127.0.191.254:36031: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
I20260926 06:56:15.485901 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:15.485787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:37685)
0926 06:56:15.485843 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:15.485875 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 06:56:15.486199 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:15.486121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:38495)
0926 06:56:15.486161 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:15.486184 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 06:56:15.486316 17665 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 06:56:16.300719 17669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:16.300457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:34549)
0926 06:56:16.300609 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:16.300653 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260926 06:56:16.357808 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:16.357600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:50995)
0926 06:56:16.357717 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:16.357756 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":53,"threads_started":1}
I20260926 06:56:16.486819 17671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:16.486626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:44787)
0926 06:56:16.486744 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:16.486773 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260926 06:56:16.507226 17672 raft_consensus.cc:482] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:56:16.507416 17672 raft_consensus.cc:504] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:56:16.507753 17672 leader_election.cc:288] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299), 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177)
Sep 26 06:56:16 dist-test-slave-ws74 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.195@KRBTEST.COM
Sep 26 06:56:16 dist-test-slave-ws74 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.194@KRBTEST.COM
I20260926 06:56:16.518059 17669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:16.507944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:42177 (local address 127.0.191.193:37533)
0926 06:56:16.508004 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:16.508017 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:16.508086 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:16.508999 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:16.509001 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:16.509502 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:16.509683 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:16.509689 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:16.510753 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:16.510754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:16.511530 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:16.511532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:16.512092 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:16.512093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:16.512230 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:16.512860 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:56:16.512864 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:16.514616 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:16.516645 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:16.516646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:16.516647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:16.516886 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:16.517128 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:16.517129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:16.517130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:16.517218 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:16.517680 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:16.517685 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:16.517791 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 06:56:16.517881 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 06:56:16.517992 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":8}
I20260926 06:56:16.518111 17675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:16.508119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:37533 (local address 127.0.191.194:42177)
0926 06:56:16.508249 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:16.508260 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:16.508296 (+ 36us) server_negotiation.cc:407] Connection header received
0926 06:56:16.508778 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:16.508780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:16.508817 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:16.508966 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:16.509801 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:16.510641 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:16.511617 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:16.512001 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:16.512122 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:16.514734 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:16.514737 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:16.514737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:16.514746 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:16.516573 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:16.516958 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:16.516961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:16.516961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:16.517057 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:16.517314 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:16.517315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:16.517315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:16.517605 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:16.517692 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:16.517937 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 06:56:16.518036 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260926 06:56:16.518469 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:16.507943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:40299 (local address 127.0.191.193:58599)
0926 06:56:16.508173 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:16.508184 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:16.508306 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:16.508999 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:16.509002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:16.509503 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:16.509683 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:16.509689 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:16.510764 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:16.510765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:16.511529 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:16.511531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:16.512107 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:16.512108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:16.512230 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:16.512790 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:56:16.512798 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:16.514227 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:16.516450 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:16.516456 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:16.516458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:16.516747 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:16.517004 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:16.517004 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:16.517004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:16.517155 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:16.518170 (+ 1015us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:16.518171 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:16.518217 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 06:56:16.518303 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 06:56:16.518390 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":177,"thread_start_us":53,"threads_started":1}
I20260926 06:56:16.518766 17674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:16.508070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:58599 (local address 127.0.191.195:40299)
0926 06:56:16.508208 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:16.508221 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:16.508259 (+ 38us) server_negotiation.cc:407] Connection header received
0926 06:56:16.508768 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:16.508770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:16.508812 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:16.508981 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:16.509797 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:16.510662 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:16.511632 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:16.512017 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:16.512127 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:16.514335 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:16.514339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:16.514340 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:16.514352 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:16.516367 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:16.516826 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:16.516833 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:16.516834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:16.516934 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:16.517215 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:16.517217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:16.517217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:16.517605 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:16.517692 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:16.518531 (+ 839us) server_negotiation.cc:299] Negotiation successful
0926 06:56:16.518658 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
W20260926 06:56:16.518975 17287 leader_election.cc:341] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177): Not found: Tablet not found: a747382066e24fa9924e168a652eec29
W20260926 06:56:16.519366 17286 leader_election.cc:341] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299): Not found: Tablet not found: a747382066e24fa9924e168a652eec29
I20260926 06:56:16.519423 17286 leader_election.cc:302] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 04a5c1e9863e481387554126e1f6383a; no voters: 00a0eeeabef14875aee59035e4cf1526, 460f16e281384d10aa8a29107c8665a0
I20260926 06:56:16.519531 17672 raft_consensus.cc:2726] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:56:17.301522 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:17.301215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:57563)
0926 06:56:17.301433 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:17.301474 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":63,"threads_started":1}
I20260926 06:56:17.358518 17677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:17.358293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:48985)
0926 06:56:17.358432 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:17.358477 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260926 06:56:17.487426 17678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:17.487205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:47823)
0926 06:56:17.487348 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:17.487385 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":56,"threads_started":1}
I20260926 06:56:18.302275 17679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:18.302008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:58275)
0926 06:56:18.302171 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:18.302221 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":77,"threads_started":1}
I20260926 06:56:18.359251 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:18.359069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:43385)
0926 06:56:18.359181 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:18.359210 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260926 06:56:18.488019 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:18.487821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:34283)
0926 06:56:18.487938 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:18.487979 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260926 06:56:18.631363 17682 raft_consensus.cc:482] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:56:18.631518 17682 raft_consensus.cc:504] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:56:18.631721 17682 leader_election.cc:288] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299), 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177)
W20260926 06:56:18.632097 17287 leader_election.cc:341] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177): Not found: Tablet not found: a747382066e24fa9924e168a652eec29
W20260926 06:56:18.632179 17286 leader_election.cc:341] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299): Not found: Tablet not found: a747382066e24fa9924e168a652eec29
I20260926 06:56:18.632256 17286 leader_election.cc:302] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 04a5c1e9863e481387554126e1f6383a; no voters: 00a0eeeabef14875aee59035e4cf1526, 460f16e281384d10aa8a29107c8665a0
I20260926 06:56:18.632333 17682 raft_consensus.cc:2726] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:56:19.303023 17683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:19.302790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:56117)
0926 06:56:19.302919 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:19.302963 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260926 06:56:19.359853 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:19.359659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:58229)
0926 06:56:19.359773 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:19.359810 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260926 06:56:19.488790 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:19.488482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:46347)
0926 06:56:19.488703 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:19.488747 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":72,"threads_started":1}
I20260926 06:56:20.303853 17686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:20.303586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:60675)
0926 06:56:20.303764 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:20.303804 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":78,"threads_started":1}
I20260926 06:56:20.360468 17687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:20.360275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:33851)
0926 06:56:20.360390 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:20.360429 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260926 06:56:20.489462 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:20.489221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:60967)
0926 06:56:20.489345 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:20.489423 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260926 06:56:21.304690 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:21.304435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:42871)
0926 06:56:21.304597 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:21.304642 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260926 06:56:21.327183 17690 raft_consensus.cc:482] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:56:21.327268 17690 raft_consensus.cc:504] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:56:21.327419 17690 leader_election.cc:288] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299), 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177)
W20260926 06:56:21.327757 17286 leader_election.cc:341] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299): Not found: Tablet not found: a747382066e24fa9924e168a652eec29
W20260926 06:56:21.327839 17287 leader_election.cc:341] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177): Not found: Tablet not found: a747382066e24fa9924e168a652eec29
I20260926 06:56:21.327922 17287 leader_election.cc:302] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 04a5c1e9863e481387554126e1f6383a; no voters: 00a0eeeabef14875aee59035e4cf1526, 460f16e281384d10aa8a29107c8665a0
I20260926 06:56:21.328011 17690 raft_consensus.cc:2726] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:56:21.361106 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:21.360912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:33063)
0926 06:56:21.361027 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:21.361066 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260926 06:56:21.490113 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:21.489896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:45861)
0926 06:56:21.490029 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:21.490068 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260926 06:56:22.305491 17693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:22.305171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:60679)
0926 06:56:22.305367 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:22.305446 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":84,"threads_started":1}
I20260926 06:56:22.361743 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:22.361549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:57479)
0926 06:56:22.361668 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:22.361704 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":57,"threads_started":1}
I20260926 06:56:22.490759 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:22.490547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:49195)
0926 06:56:22.490660 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:22.490700 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260926 06:56:23.306160 17696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:23.305950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:42025)
0926 06:56:23.306079 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:23.306114 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260926 06:56:23.362391 17697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:23.362193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:35897)
0926 06:56:23.362307 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:23.362346 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260926 06:56:23.491456 17698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:23.491225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:58745)
0926 06:56:23.491362 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:23.491409 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":58,"threads_started":1}
I20260926 06:56:24.306851 17699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:24.306636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:38527)
0926 06:56:24.306768 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:24.306806 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
I20260926 06:56:24.363058 17700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:24.362852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:60895)
0926 06:56:24.362973 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:24.363013 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260926 06:56:24.492147 17701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:24.491929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:36401)
0926 06:56:24.492071 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:24.492104 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260926 06:56:25.307613 17702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:25.307374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:45607)
0926 06:56:25.307521 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:25.307562 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":74,"threads_started":1}
I20260926 06:56:25.363682 17703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:25.363463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:59381)
0926 06:56:25.363605 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:25.363640 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260926 06:56:25.492897 17704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:25.492616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:45789)
0926 06:56:25.492814 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:25.492853 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":121,"threads_started":1}
I20260926 06:56:26.093415 17705 raft_consensus.cc:482] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:56:26.093531 17705 raft_consensus.cc:504] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f16e281384d10aa8a29107c8665a0" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 40299 } } peers { permanent_uuid: "00a0eeeabef14875aee59035e4cf1526" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42177 } } peers { permanent_uuid: "04a5c1e9863e481387554126e1f6383a" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 33491 } }
I20260926 06:56:26.093758 17705 leader_election.cc:288] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299), 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177)
W20260926 06:56:26.094127 17286 leader_election.cc:341] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 460f16e281384d10aa8a29107c8665a0 (127.0.191.195:40299): Not found: Tablet not found: a747382066e24fa9924e168a652eec29
W20260926 06:56:26.094228 17287 leader_election.cc:341] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 00a0eeeabef14875aee59035e4cf1526 (127.0.191.194:42177): Not found: Tablet not found: a747382066e24fa9924e168a652eec29
I20260926 06:56:26.094266 17287 leader_election.cc:302] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 04a5c1e9863e481387554126e1f6383a; no voters: 00a0eeeabef14875aee59035e4cf1526, 460f16e281384d10aa8a29107c8665a0
I20260926 06:56:26.094365 17705 raft_consensus.cc:2726] T a747382066e24fa9924e168a652eec29 P 04a5c1e9863e481387554126e1f6383a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:56:26.308259 17706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:26.308046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:58991)
0926 06:56:26.308172 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:26.308210 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":59,"threads_started":1}
I20260926 06:56:26.364281 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:26.364095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:47475)
0926 06:56:26.364208 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:26.364240 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260926 06:56:26.493625 17708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:26.493360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:32893)
0926 06:56:26.493526 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:26.493568 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":110,"threads_started":1}
I20260926 06:56:27.309000 17709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:27.308763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:49699)
0926 06:56:27.308904 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:27.308940 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260926 06:56:27.364959 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:27.364740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:51393)
0926 06:56:27.364875 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:27.364912 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260926 06:56:27.494284 17711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:27.494066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:45551)
0926 06:56:27.494194 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:27.494236 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":58,"threads_started":1}
I20260926 06:56:28.309770 17712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:28.309538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:35993)
0926 06:56:28.309649 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:28.309707 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":63,"threads_started":1}
I20260926 06:56:28.365751 17713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:28.365552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:49429)
0926 06:56:28.365668 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:28.365707 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260926 06:56:28.495018 17714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:28.494786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:40225)
0926 06:56:28.494929 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:28.494959 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260926 06:56:29.310392 17715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:29.310171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:54675)
0926 06:56:29.310310 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:29.310346 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":62,"threads_started":1}
I20260926 06:56:29.366397 17716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:29.366155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:33055)
0926 06:56:29.366310 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:29.366351 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260926 06:56:29.495754 17717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:29.495521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:55403)
0926 06:56:29.495660 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:29.495703 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":61,"threads_started":1}
I20260926 06:56:30.311204 17718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:30.310929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.193:50531)
0926 06:56:30.311117 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:30.311159 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":109,"threads_started":1}
I20260926 06:56:30.367108 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:30.366915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.194:48337)
0926 06:56:30.367032 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:30.367067 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260926 06:56:30.493646 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 17235
I20260926 06:56:30.496516 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:30.496320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36031 (local address 127.0.191.195:57307)
0926 06:56:30.496430 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:30.496469 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.254:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":67,"threads_started":1}
I20260926 06:56:30.498544 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 17403
I20260926 06:56:30.503011 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 17536
2026-09-26T06:56:30Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58175 ms)
[----------] 30 tests from SecurityITest (231031 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:30 dist-test-slave-ws74 krb5kdc[17723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:30 dist-test-slave-ws74 krb5kdc[17723](info): set up 2 sockets
Sep 26 06:56:30 dist-test-slave-ws74 krb5kdc[17723](info): commencing operation
krb5kdc: starting...
W20260926 06:56:31.609090 767 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:31 dist-test-slave-ws74 krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:56:31.728346 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:42517
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:44369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:42517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:31.811010 17736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:31.811213 17736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:31.811239 17736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:31.812294 17736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:31.812326 17736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:31.812340 17736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:31.812352 17736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:31.812364 17736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:31.813791 17736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:44369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:42517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:42517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:31.813973 17736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:31.814128 17736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:31.819310 17742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:31.819382 17744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:31.819422 17736 server_base.cc:1034] running on GCE node
W20260926 06:56:31.819346 17741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:31.819733 17736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:31.820016 17736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:31.821198 17736 hybrid_clock.cc:648] HybridClock initialized: now 1790405791821175 us; error 51 us; skew 500 ppm
Sep 26 06:56:31 dist-test-slave-ws74 krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:31.823654 17736 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:56:31.824062 17736 webserver.cc:460] Webserver started at http://127.0.191.254:38597/ using document root <none> and password file <none>
I20260926 06:56:31.824290 17736 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:31.824333 17736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:31.824501 17736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:31.825594 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "296cd86702a2499a81ffc031ec18aea5"
format_stamp: "Formatted at 2026-09-26 06:56:31 on dist-test-slave-ws74"
server_key: "ba7f12837a47e437a7420c2e08d326ca"
server_key_iv: "bf8b725599d83401ce15fe8f3aef5334"
server_key_version: "encryptionkey@0"
I20260926 06:56:31.825944 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "296cd86702a2499a81ffc031ec18aea5"
format_stamp: "Formatted at 2026-09-26 06:56:31 on dist-test-slave-ws74"
server_key: "ba7f12837a47e437a7420c2e08d326ca"
server_key_iv: "bf8b725599d83401ce15fe8f3aef5334"
server_key_version: "encryptionkey@0"
I20260926 06:56:31.827231 17736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 06:56:31.827937 17751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:31.828109 17736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:31.828193 17736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "296cd86702a2499a81ffc031ec18aea5"
format_stamp: "Formatted at 2026-09-26 06:56:31 on dist-test-slave-ws74"
server_key: "ba7f12837a47e437a7420c2e08d326ca"
server_key_iv: "bf8b725599d83401ce15fe8f3aef5334"
server_key_version: "encryptionkey@0"
I20260926 06:56:31.828253 17736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:31.848933 17736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:31.850703 17736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:31.850808 17736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:31.854503 17736 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:42517
I20260926 06:56:31.854566 17803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:42517 every 8 connection(s)
I20260926 06:56:31.854894 17736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:31.855743 17804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:31.858009 17804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5: Bootstrap starting.
I20260926 06:56:31.858512 17804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:31.858770 17804 log.cc:824] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:31.859388 17804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5: No bootstrap required, opened a new log
I20260926 06:56:31.860841 17804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "296cd86702a2499a81ffc031ec18aea5" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42517 } }
I20260926 06:56:31.860984 17804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:31.861008 17804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 296cd86702a2499a81ffc031ec18aea5, State: Initialized, Role: FOLLOWER
I20260926 06:56:31.861155 17804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "296cd86702a2499a81ffc031ec18aea5" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42517 } }
I20260926 06:56:31.861238 17804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:31.861261 17804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:31.861289 17804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:31.861768 17804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "296cd86702a2499a81ffc031ec18aea5" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42517 } }
I20260926 06:56:31.861853 17804 leader_election.cc:302] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 296cd86702a2499a81ffc031ec18aea5; no voters:
I20260926 06:56:31.861980 17804 leader_election.cc:288] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:31.862021 17807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:31.862187 17807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [term 1 LEADER]: Becoming Leader. State: Replica: 296cd86702a2499a81ffc031ec18aea5, State: Running, Role: LEADER
I20260926 06:56:31.862257 17804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:31.862324 17807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "296cd86702a2499a81ffc031ec18aea5" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42517 } }
I20260926 06:56:31.862733 17809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 296cd86702a2499a81ffc031ec18aea5. Latest consensus state: current_term: 1 leader_uuid: "296cd86702a2499a81ffc031ec18aea5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "296cd86702a2499a81ffc031ec18aea5" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42517 } } }
I20260926 06:56:31.862870 17808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "296cd86702a2499a81ffc031ec18aea5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "296cd86702a2499a81ffc031ec18aea5" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 42517 } } }
I20260926 06:56:31.862890 17809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:31.863022 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 17736
I20260926 06:56:31.863137 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:31.862962 17808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:31.863318 17813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:31.863369 767 external_mini_cluster.cc:1428] Setting key 905538a9506dce1d8d68260422f90ce0
I20260926 06:56:31.863850 17813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:31.865644 17813 catalog_manager.cc:1347] Generated new cluster ID: cfe12edffae94df58e6bbd19f4384e80
I20260926 06:56:31.865696 17813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:56:31 dist-test-slave-ws74 krb5kdc[17723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:31.873907 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:31.864029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45160 (local address 127.0.191.254:42517)
0926 06:56:31.864198 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:31.864207 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:31.865106 (+ 899us) server_negotiation.cc:407] Connection header received
0926 06:56:31.865272 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:31.865280 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:31.865336 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:31.865499 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:31.866165 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:31.867055 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:31.867817 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:31.868266 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:31.868397 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:31.870748 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:31.870752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:31.870752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:31.870768 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:31.872389 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:31.872743 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:31.872746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:31.872746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:31.872840 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:31.873045 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:31.873046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:31.873047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:31.873272 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:31.873450 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:31.873569 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 06:56:31.873707 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260926 06:56:31.894049 17813 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:31.894613 17813 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:31.900290 17813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5: Generated new TSK 0
I20260926 06:56:31.900487 17813 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:56:31.934244 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42517
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:44369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:32.040454 17830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:32.040680 17830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:32.040705 17830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:32.042011 17830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:32.042057 17830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:32.042085 17830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:56:32.043640 17830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:44369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:32.043915 17830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:32.044154 17830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:32.049206 17836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:32.049206 17835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:32.049623 17830 server_base.cc:1034] running on GCE node
W20260926 06:56:32.049223 17838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:32.049902 17830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:32.050158 17830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:32.051335 17830 hybrid_clock.cc:648] HybridClock initialized: now 1790405792051322 us; error 34 us; skew 500 ppm
Sep 26 06:56:32 dist-test-slave-ws74 krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:32.053931 17830 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:56:32.054387 17830 webserver.cc:460] Webserver started at http://127.0.191.193:35383/ using document root <none> and password file <none>
I20260926 06:56:32.054623 17830 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:32.054693 17830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:32.054843 17830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:32.055882 17830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "60aef343e7bd4afdbb1f59f823d8ce2c"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "9570927007dfcd38cdbf869f243604a5"
server_key_iv: "b5191fc2f6a066c5c51e2e68f1e2c7b2"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.056253 17830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "60aef343e7bd4afdbb1f59f823d8ce2c"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "9570927007dfcd38cdbf869f243604a5"
server_key_iv: "b5191fc2f6a066c5c51e2e68f1e2c7b2"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.057672 17830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:56:32.058476 17845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.058760 17830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:32.058872 17830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "60aef343e7bd4afdbb1f59f823d8ce2c"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "9570927007dfcd38cdbf869f243604a5"
server_key_iv: "b5191fc2f6a066c5c51e2e68f1e2c7b2"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.058950 17830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:32.070719 17830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:32.072535 17830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:32.072698 17830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:32.072926 17830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:32.073226 17830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:32.073282 17830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.073316 17830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:32.073343 17830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.079677 17830 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:39105
I20260926 06:56:32.079730 17958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:39105 every 8 connection(s)
I20260926 06:56:32.080078 17830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:56:32.085235 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 17830
I20260926 06:56:32.085412 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:56:32.085620 767 external_mini_cluster.cc:1428] Setting key bf5ab85a2df5e712e795acb50e1c2e8f
Sep 26 06:56:32 dist-test-slave-ws74 krb5kdc[17723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:32.090293 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.080837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:43305 (local address 127.0.191.254:42517)
0926 06:56:32.080926 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:32.080928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:32.081093 (+ 165us) server_negotiation.cc:407] Connection header received
0926 06:56:32.082069 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:32.082070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:32.082084 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:32.082144 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:32.082738 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.083236 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.084312 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.084508 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.084565 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.086947 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:32.086949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:32.086949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:32.086956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:32.088656 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:32.089212 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:32.089213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:32.089213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:32.089268 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:32.089607 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:32.089608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:32.089608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:32.089714 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:32.089769 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:32.090153 (+ 384us) server_negotiation.cc:299] Negotiation successful
0926 06:56:32.090229 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 06:56:32.090597 17961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.080803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:42517 (local address 127.0.191.193:43305)
0926 06:56:32.080964 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:32.081006 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:32.081977 (+ 971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:32.082217 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:32.082220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:32.082478 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:32.082663 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.082667 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.083357 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.083358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.084207 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.084209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.084585 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.084585 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.084685 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.085253 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:56:32.085261 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:32.086858 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:32.088751 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:32.088759 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:32.088762 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:32.089124 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:32.089349 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:32.089351 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:32.089351 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:32.089534 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:32.089788 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:32.089795 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:32.089980 (+ 185us) client_negotiation.cc:769] Sending connection context
0926 06:56:32.090107 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 06:56:32.090309 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260926 06:56:32.091120 17959 heartbeater.cc:347] Connected to a master server at 127.0.191.254:42517
I20260926 06:56:32.091230 17959 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:32.091408 17959 heartbeater.cc:511] Master 127.0.191.254:42517 requested a full tablet report, sending...
I20260926 06:56:32.093175 17768 ts_manager.cc:194] Registered new tserver with Master: 60aef343e7bd4afdbb1f59f823d8ce2c (127.0.191.193:39105)
I20260926 06:56:32.093848 17768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:43305
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:56:32.112223 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42517
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:44369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:32.192900 17964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:32.193114 17964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:32.193138 17964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:32.194242 17964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:32.194278 17964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:32.194303 17964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:56:32.195685 17964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:44369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:32.195860 17964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:32.196043 17964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:32.200927 17970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:32.200927 17969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:32.200932 17972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:32.200994 17964 server_base.cc:1034] running on GCE node
I20260926 06:56:32.201516 17964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:32.201716 17964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:32.202855 17964 hybrid_clock.cc:648] HybridClock initialized: now 1790405792202845 us; error 31 us; skew 500 ppm
Sep 26 06:56:32 dist-test-slave-ws74 krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:32.205236 17964 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:56:32.205765 17964 webserver.cc:460] Webserver started at http://127.0.191.194:35281/ using document root <none> and password file <none>
I20260926 06:56:32.205991 17964 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:32.206059 17964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:32.206212 17964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:32.207247 17964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "e7f139dc22d444ea89a20b9e59948431"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "7876de7e8b638266f010ae82d20bbbc8"
server_key_iv: "8c9b253be261dfdcc386da555ec46c64"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.207615 17964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "e7f139dc22d444ea89a20b9e59948431"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "7876de7e8b638266f010ae82d20bbbc8"
server_key_iv: "8c9b253be261dfdcc386da555ec46c64"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.208892 17964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:56:32.209633 17979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.209800 17964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:32.209896 17964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "e7f139dc22d444ea89a20b9e59948431"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "7876de7e8b638266f010ae82d20bbbc8"
server_key_iv: "8c9b253be261dfdcc386da555ec46c64"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.209975 17964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:32.227697 17964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:32.229616 17964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:32.229770 17964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:32.229991 17964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:32.230350 17964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:32.230403 17964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.230443 17964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:32.230469 17964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.236392 17964 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:33801
I20260926 06:56:32.236433 18092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:33801 every 8 connection(s)
I20260926 06:56:32.236783 17964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:56:32 dist-test-slave-ws74 krb5kdc[17723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:32.246141 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.237520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:44367 (local address 127.0.191.254:42517)
0926 06:56:32.237581 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:32.237583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:32.237810 (+ 227us) server_negotiation.cc:407] Connection header received
0926 06:56:32.238791 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:32.238792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:32.238802 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:32.238855 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:32.239471 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.239964 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.241000 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.241178 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.241226 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.243550 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:32.243552 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:32.243552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:32.243558 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:32.244768 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:32.245152 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:32.245153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:32.245153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:32.245201 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:32.245525 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:32.245526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:32.245527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:32.245674 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:32.245750 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:32.246004 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 06:56:32.246074 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:56:32.246296 18095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.237516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:42517 (local address 127.0.191.194:44367)
0926 06:56:32.237701 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:32.237740 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:32.238694 (+ 954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:32.238915 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:32.238918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:32.239188 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:32.239395 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.239400 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.240044 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.240044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.240926 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.240928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.241249 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.241250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.241354 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.241998 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:56:32.242023 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:32.243466 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:32.244853 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:32.244859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:32.244861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:32.245086 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:32.245268 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:32.245269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:32.245269 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:32.245444 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:32.245743 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:32.245747 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:32.245880 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 06:56:32.245963 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 06:56:32.246087 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260926 06:56:32.246852 18093 heartbeater.cc:347] Connected to a master server at 127.0.191.254:42517
I20260926 06:56:32.246958 18093 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:32.247054 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 17964
I20260926 06:56:32.247169 18093 heartbeater.cc:511] Master 127.0.191.254:42517 requested a full tablet report, sending...
I20260926 06:56:32.247184 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:56:32.247442 767 external_mini_cluster.cc:1428] Setting key 525cf454a149a84cda3a84a8f82191e2
I20260926 06:56:32.247591 17768 ts_manager.cc:194] Registered new tserver with Master: e7f139dc22d444ea89a20b9e59948431 (127.0.191.194:33801)
I20260926 06:56:32.248245 17768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:44367
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:56:32.272424 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42517
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:44369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:32.353132 18098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:32.353328 18098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:32.353351 18098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:32.354526 18098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:32.354571 18098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:32.354598 18098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:56:32.356056 18098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:44369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:42517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:32.356290 18098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:32.356564 18098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:56:32.361656 18098 server_base.cc:1034] running on GCE node
W20260926 06:56:32.361603 18106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:32.361624 18104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:32.361600 18103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:32.362200 18098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:32.362489 18098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:32.363654 18098 hybrid_clock.cc:648] HybridClock initialized: now 1790405792363639 us; error 34 us; skew 500 ppm
Sep 26 06:56:32 dist-test-slave-ws74 krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:32.366163 18098 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:56:32.366530 18098 webserver.cc:460] Webserver started at http://127.0.191.195:41689/ using document root <none> and password file <none>
I20260926 06:56:32.366755 18098 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:32.366799 18098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:32.366962 18098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:32.367975 18098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "1d6a9a7bdc2348d490f507f28f99e84a"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "32f0c1810c2e6e24a971e4b335b7f4fb"
server_key_iv: "e4abf8fce463e814c0699567fc1dfb3f"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.368322 18098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "1d6a9a7bdc2348d490f507f28f99e84a"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "32f0c1810c2e6e24a971e4b335b7f4fb"
server_key_iv: "e4abf8fce463e814c0699567fc1dfb3f"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.369676 18098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:56:32.370368 18113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.370532 18098 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.370595 18098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "1d6a9a7bdc2348d490f507f28f99e84a"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "32f0c1810c2e6e24a971e4b335b7f4fb"
server_key_iv: "e4abf8fce463e814c0699567fc1dfb3f"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.370682 18098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:32.381101 18098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:32.383342 18098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:32.383495 18098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:32.383724 18098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:32.384011 18098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:32.384044 18098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.384097 18098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:32.384125 18098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.389830 18098 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:33471
I20260926 06:56:32.389912 18226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:33471 every 8 connection(s)
I20260926 06:56:32.390198 18098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:56:32 dist-test-slave-ws74 krb5kdc[17723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:32.397294 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 18098
I20260926 06:56:32.397423 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:56:32.397598 767 external_mini_cluster.cc:1428] Setting key 18daebab2604440e835bce991f9dded1
I20260926 06:56:32.399821 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.390943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:41399 (local address 127.0.191.254:42517)
0926 06:56:32.391072 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:32.391075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:32.391246 (+ 171us) server_negotiation.cc:407] Connection header received
0926 06:56:32.392218 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:32.392220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:32.392234 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:32.392280 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:32.392948 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.393556 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.394583 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.394780 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.394891 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.396955 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:32.396956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:32.396957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:32.396965 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:32.398769 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:32.399125 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:32.399126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:32.399126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:32.399176 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:32.399343 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:32.399344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:32.399344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:32.399440 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:32.399501 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:32.399701 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 06:56:32.399759 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260926 06:56:32.400027 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.390896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:42517 (local address 127.0.191.195:41399)
0926 06:56:32.391106 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:32.391150 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:32.392099 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:32.392364 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:32.392368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:32.392647 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:32.392849 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.392854 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.393647 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.393648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.394474 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.394476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.394898 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.394899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.395027 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.395475 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:56:32.395483 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:32.396852 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:32.398842 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:32.398848 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:32.398850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:32.399060 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:32.399215 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:32.399215 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:32.399216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:32.399296 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:32.399491 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:32.399493 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:32.399596 (+ 103us) client_negotiation.cc:769] Sending connection context
0926 06:56:32.399674 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 06:56:32.399800 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
I20260926 06:56:32.400482 18227 heartbeater.cc:347] Connected to a master server at 127.0.191.254:42517
I20260926 06:56:32.400561 18227 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:32.400766 18227 heartbeater.cc:511] Master 127.0.191.254:42517 requested a full tablet report, sending...
I20260926 06:56:32.401157 17768 ts_manager.cc:194] Registered new tserver with Master: 1d6a9a7bdc2348d490f507f28f99e84a (127.0.191.195:33471)
I20260926 06:56:32.401587 17768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:41399
I20260926 06:56:32.409512 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:56:32.416364 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.410828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45172 (local address 127.0.191.254:42517)
0926 06:56:32.410895 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:32.410897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:32.410952 (+ 55us) server_negotiation.cc:407] Connection header received
0926 06:56:32.410993 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:32.410993 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:32.411002 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:32.411047 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:32.411532 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.411991 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.412784 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.412967 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.413014 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.414252 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:32.414253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:32.414253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:32.414258 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:32.415419 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:32.415734 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:32.415735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:32.415735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:32.415783 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:32.415929 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:32.415930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:32.415930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:32.416071 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:32.416148 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:32.416247 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 06:56:32.416304 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:56:32.418383 17768 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:56:32.419910 17768 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:56:32.427052 18239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.422412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39105 (local address 127.0.0.1:59290)
0926 06:56:32.422637 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:32.422650 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:32.422716 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:32.423337 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:32.423338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:32.423340 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:32.423398 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.423400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.424613 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.424614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.426048 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.426051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.426736 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.426737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.426815 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.426828 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 06:56:32.426896 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 06:56:32.426954 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":182,"thread_start_us":82,"threads_started":1}
I20260926 06:56:32.427290 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.422562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.0.191.193:39105)
0926 06:56:32.422957 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:32.422962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:32.423063 (+ 101us) server_negotiation.cc:407] Connection header received
0926 06:56:32.423112 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:32.423115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:32.423133 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:32.423218 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:32.423470 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.424510 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.426149 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.426634 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.426732 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.426793 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:32.427165 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 06:56:32.427215 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":99,"threads_started":1}
I20260926 06:56:32.427343 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.422602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:33801 (local address 127.0.0.1:50454)
0926 06:56:32.422760 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:32.422771 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:32.422830 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:32.423284 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:32.423285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:32.423287 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:32.423329 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.423330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.424650 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.424651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.425938 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.425941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.427014 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.427015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.427075 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.427079 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:56:32.427226 (+ 147us) client_negotiation.cc:240] Negotiation successful
0926 06:56:32.427268 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260926 06:56:32.427570 18243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.422848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50454 (local address 127.0.191.194:33801)
0926 06:56:32.423042 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:32.423046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:32.423056 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:56:32.423093 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:32.423095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:32.423110 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:32.423182 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:32.423397 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.424493 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.426030 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.426935 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.427011 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.427064 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:32.427444 (+ 380us) server_negotiation.cc:299] Negotiation successful
0926 06:56:32.427492 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260926 06:56:32.427675 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.422370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:33471 (local address 127.0.0.1:38526)
0926 06:56:32.422602 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:32.422648 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:32.422716 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:32.423113 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:32.423116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:32.423118 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:32.423184 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.423185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.424497 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.424498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.426048 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.426051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.427077 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.427078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.427155 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.427160 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:56:32.427218 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 06:56:32.427259 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":137,"thread_start_us":81,"threads_started":1}
I20260926 06:56:32.427922 18240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.422487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38526 (local address 127.0.191.195:33471)
0926 06:56:32.422851 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:32.422856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:32.422872 (+ 16us) server_negotiation.cc:407] Connection header received
0926 06:56:32.422901 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:32.422903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:32.422915 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:32.423018 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:32.423282 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.424376 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.426142 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.426892 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.426974 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.427037 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:32.427786 (+ 749us) server_negotiation.cc:299] Negotiation successful
0926 06:56:32.427833 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":75,"threads_started":1}
I20260926 06:56:32.428050 17893 tablet_service.cc:1467] Processing CreateTablet for tablet 1608eefb3e704ccca93af09e27fedc8f (DEFAULT_TABLE table=test-table [id=9fba54a4be144c6db8646f75f5294f06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:32.428341 17893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1608eefb3e704ccca93af09e27fedc8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:32.428262 18027 tablet_service.cc:1467] Processing CreateTablet for tablet 1608eefb3e704ccca93af09e27fedc8f (DEFAULT_TABLE table=test-table [id=9fba54a4be144c6db8646f75f5294f06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:32.428524 18027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1608eefb3e704ccca93af09e27fedc8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:32.431057 18244 tablet_bootstrap.cc:492] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c: Bootstrap starting.
I20260926 06:56:32.431581 18245 tablet_bootstrap.cc:492] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431: Bootstrap starting.
I20260926 06:56:32.431826 18244 tablet_bootstrap.cc:654] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:32.432273 18245 tablet_bootstrap.cc:654] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:32.432350 18161 tablet_service.cc:1467] Processing CreateTablet for tablet 1608eefb3e704ccca93af09e27fedc8f (DEFAULT_TABLE table=test-table [id=9fba54a4be144c6db8646f75f5294f06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:32.432502 18244 log.cc:824] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:32.432741 18161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1608eefb3e704ccca93af09e27fedc8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:32.432858 18245 log.cc:824] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:32.433350 18244 tablet_bootstrap.cc:492] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c: No bootstrap required, opened a new log
I20260926 06:56:32.433492 18244 ts_tablet_manager.cc:1397] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 06:56:32.433997 18245 tablet_bootstrap.cc:492] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431: No bootstrap required, opened a new log
I20260926 06:56:32.434135 18245 ts_tablet_manager.cc:1397] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 06:56:32.435180 18244 raft_consensus.cc:348] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 33471 } } peers { permanent_uuid: "e7f139dc22d444ea89a20b9e59948431" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33801 } } peers { permanent_uuid: "60aef343e7bd4afdbb1f59f823d8ce2c" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39105 } }
I20260926 06:56:32.435380 18244 raft_consensus.cc:374] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:32.435451 18244 raft_consensus.cc:729] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60aef343e7bd4afdbb1f59f823d8ce2c, State: Initialized, Role: FOLLOWER
I20260926 06:56:32.435588 18244 consensus_queue.cc:260] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 33471 } } peers { permanent_uuid: "e7f139dc22d444ea89a20b9e59948431" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33801 } } peers { permanent_uuid: "60aef343e7bd4afdbb1f59f823d8ce2c" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39105 } }
I20260926 06:56:32.435905 18244 ts_tablet_manager.cc:1428] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:56:32.436017 18249 tablet_bootstrap.cc:492] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a: Bootstrap starting.
I20260926 06:56:32.436203 17959 heartbeater.cc:503] Master 127.0.191.254:42517 was elected leader, sending a full tablet report...
I20260926 06:56:32.436272 18245 raft_consensus.cc:348] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 33471 } } peers { permanent_uuid: "e7f139dc22d444ea89a20b9e59948431" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33801 } } peers { permanent_uuid: "60aef343e7bd4afdbb1f59f823d8ce2c" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39105 } }
I20260926 06:56:32.436436 18245 raft_consensus.cc:374] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:32.436480 18245 raft_consensus.cc:729] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7f139dc22d444ea89a20b9e59948431, State: Initialized, Role: FOLLOWER
I20260926 06:56:32.436630 18245 consensus_queue.cc:260] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 33471 } } peers { permanent_uuid: "e7f139dc22d444ea89a20b9e59948431" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33801 } } peers { permanent_uuid: "60aef343e7bd4afdbb1f59f823d8ce2c" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39105 } }
I20260926 06:56:32.436761 18249 tablet_bootstrap.cc:654] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:32.436902 18245 ts_tablet_manager.cc:1428] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:56:32.436939 18093 heartbeater.cc:503] Master 127.0.191.254:42517 was elected leader, sending a full tablet report...
I20260926 06:56:32.437536 18249 log.cc:824] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:32.438333 18249 tablet_bootstrap.cc:492] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a: No bootstrap required, opened a new log
I20260926 06:56:32.438411 18249 ts_tablet_manager.cc:1397] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:56:32.439694 18249 raft_consensus.cc:348] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 33471 } } peers { permanent_uuid: "e7f139dc22d444ea89a20b9e59948431" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33801 } } peers { permanent_uuid: "60aef343e7bd4afdbb1f59f823d8ce2c" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39105 } }
I20260926 06:56:32.439844 18249 raft_consensus.cc:374] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:32.439889 18249 raft_consensus.cc:729] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d6a9a7bdc2348d490f507f28f99e84a, State: Initialized, Role: FOLLOWER
I20260926 06:56:32.440003 18249 consensus_queue.cc:260] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 33471 } } peers { permanent_uuid: "e7f139dc22d444ea89a20b9e59948431" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33801 } } peers { permanent_uuid: "60aef343e7bd4afdbb1f59f823d8ce2c" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39105 } }
I20260926 06:56:32.440256 18249 ts_tablet_manager.cc:1428] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:56:32.440359 18227 heartbeater.cc:503] Master 127.0.191.254:42517 was elected leader, sending a full tablet report...
I20260926 06:56:32.460309 18252 raft_consensus.cc:482] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:56:32.460421 18252 raft_consensus.cc:504] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 33471 } } peers { permanent_uuid: "e7f139dc22d444ea89a20b9e59948431" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33801 } } peers { permanent_uuid: "60aef343e7bd4afdbb1f59f823d8ce2c" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39105 } }
I20260926 06:56:32.460722 18252 leader_election.cc:288] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7f139dc22d444ea89a20b9e59948431 (127.0.191.194:33801), 60aef343e7bd4afdbb1f59f823d8ce2c (127.0.191.193:39105)
I20260926 06:56:32.464159 18243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.460983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:55499 (local address 127.0.191.194:33801)
0926 06:56:32.461038 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:32.461041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:32.461217 (+ 176us) server_negotiation.cc:407] Connection header received
0926 06:56:32.461342 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:32.461343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:32.461352 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:32.461435 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:32.461572 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.462177 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.463300 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.463824 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.463892 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.463901 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:32.464054 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 06:56:32.464091 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 06:56:32.464182 18253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.460885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:33801 (local address 127.0.191.195:55499)
0926 06:56:32.461152 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:32.461163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:32.461240 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:32.461439 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:32.461440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:32.461441 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:32.461516 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.461518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.462255 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.462256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.463199 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.463202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.463889 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.463890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.463961 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.463966 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:56:32.464046 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 06:56:32.464086 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":87,"thread_start_us":128,"threads_started":1}
I20260926 06:56:32.464249 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.461007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:34927 (local address 127.0.191.193:39105)
0926 06:56:32.461058 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:32.461060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:32.461069 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:56:32.461110 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:32.461110 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:32.461122 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:32.461197 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:32.461418 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.461992 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.463251 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.463830 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.463934 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.463943 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:32.464127 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 06:56:32.464166 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 06:56:32.464555 18047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1608eefb3e704ccca93af09e27fedc8f" candidate_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7f139dc22d444ea89a20b9e59948431" is_pre_election: true
I20260926 06:56:32.464632 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.460885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39105 (local address 127.0.191.195:34927)
0926 06:56:32.460948 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:32.460962 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:32.461026 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:32.461200 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:32.461201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:32.461202 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:32.461303 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.461323 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.462095 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.462096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.463154 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.463157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.463933 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.463935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.464035 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.464037 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:56:32.464119 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 06:56:32.464168 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 06:56:32.464713 18047 raft_consensus.cc:2454] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d6a9a7bdc2348d490f507f28f99e84a in term 0.
I20260926 06:56:32.464954 18114 leader_election.cc:302] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d6a9a7bdc2348d490f507f28f99e84a, e7f139dc22d444ea89a20b9e59948431; no voters:
I20260926 06:56:32.464965 17913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1608eefb3e704ccca93af09e27fedc8f" candidate_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60aef343e7bd4afdbb1f59f823d8ce2c" is_pre_election: true
I20260926 06:56:32.465090 17913 raft_consensus.cc:2454] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d6a9a7bdc2348d490f507f28f99e84a in term 0.
I20260926 06:56:32.465103 18252 raft_consensus.cc:2781] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:56:32.465209 18252 raft_consensus.cc:482] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:56:32.465257 18252 raft_consensus.cc:3037] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:32.466007 18252 raft_consensus.cc:504] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 33471 } } peers { permanent_uuid: "e7f139dc22d444ea89a20b9e59948431" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33801 } } peers { permanent_uuid: "60aef343e7bd4afdbb1f59f823d8ce2c" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39105 } }
I20260926 06:56:32.466152 18252 leader_election.cc:288] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [CANDIDATE]: Term 1 election: Requested vote from peers e7f139dc22d444ea89a20b9e59948431 (127.0.191.194:33801), 60aef343e7bd4afdbb1f59f823d8ce2c (127.0.191.193:39105)
I20260926 06:56:32.466290 18047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1608eefb3e704ccca93af09e27fedc8f" candidate_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7f139dc22d444ea89a20b9e59948431"
I20260926 06:56:32.466295 17913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1608eefb3e704ccca93af09e27fedc8f" candidate_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60aef343e7bd4afdbb1f59f823d8ce2c"
I20260926 06:56:32.466369 18047 raft_consensus.cc:3037] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:32.466393 17913 raft_consensus.cc:3037] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:32.467033 18047 raft_consensus.cc:2454] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d6a9a7bdc2348d490f507f28f99e84a in term 1.
I20260926 06:56:32.467155 17913 raft_consensus.cc:2454] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d6a9a7bdc2348d490f507f28f99e84a in term 1.
I20260926 06:56:32.467197 18114 leader_election.cc:302] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d6a9a7bdc2348d490f507f28f99e84a, e7f139dc22d444ea89a20b9e59948431; no voters:
I20260926 06:56:32.467281 18252 raft_consensus.cc:2781] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:32.467432 18252 raft_consensus.cc:686] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [term 1 LEADER]: Becoming Leader. State: Replica: 1d6a9a7bdc2348d490f507f28f99e84a, State: Running, Role: LEADER
I20260926 06:56:32.467536 18252 consensus_queue.cc:237] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 33471 } } peers { permanent_uuid: "e7f139dc22d444ea89a20b9e59948431" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33801 } } peers { permanent_uuid: "60aef343e7bd4afdbb1f59f823d8ce2c" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39105 } }
I20260926 06:56:32.468470 17768 catalog_manager.cc:5495] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d6a9a7bdc2348d490f507f28f99e84a (127.0.191.195). New cstate: current_term: 1 leader_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d6a9a7bdc2348d490f507f28f99e84a" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 33471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7f139dc22d444ea89a20b9e59948431" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60aef343e7bd4afdbb1f59f823d8ce2c" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39105 } health_report { overall_health: UNKNOWN } } }
I20260926 06:56:32.483776 18240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.481477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38534 (local address 127.0.191.195:33471)
0926 06:56:32.481528 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:32.481535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:32.481543 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:56:32.481573 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:32.481574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:32.481590 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:32.481652 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:56:32.481768 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.482265 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.483034 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.483232 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.483332 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.483422 (+ 90us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:56:32.483512 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:56:32.483569 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:32.483597 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 06:56:32.483613 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:56:32.485603 18047 raft_consensus.cc:1261] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431 [term 1 FOLLOWER]: Refusing update from remote peer 1d6a9a7bdc2348d490f507f28f99e84a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:56:32.485625 17913 raft_consensus.cc:1261] T 1608eefb3e704ccca93af09e27fedc8f P 60aef343e7bd4afdbb1f59f823d8ce2c [term 1 FOLLOWER]: Refusing update from remote peer 1d6a9a7bdc2348d490f507f28f99e84a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:56:32.485982 18254 consensus_queue.cc:1035] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [LEADER]: Connected to new peer: Peer: permanent_uuid: "60aef343e7bd4afdbb1f59f823d8ce2c" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:56:32.486101 18252 consensus_queue.cc:1035] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7f139dc22d444ea89a20b9e59948431" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 06:56:32.487452 18094 tablet.cc:2369] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:56:32.488545 18259 mvcc.cc:205] Tried to move back new op lower bound from 7333502126018056192 to 7333502125947875328. Current Snapshot: MvccSnapshot[applied={T|T < 7333502126018056192}]
I20260926 06:56:32.489468 18258 mvcc.cc:205] Tried to move back new op lower bound from 7333502126018056192 to 7333502125947875328. Current Snapshot: MvccSnapshot[applied={T|T < 7333502126018056192}]
I20260926 06:56:32.491106 17768 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:56:32.491182 17768 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:56:32.491876 17768 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 296cd86702a2499a81ffc031ec18aea5: Sending DeleteTablet for 3 replicas of tablet 1608eefb3e704ccca93af09e27fedc8f
I20260926 06:56:32.492187 18161 tablet_service.cc:1514] Processing DeleteTablet for tablet 1608eefb3e704ccca93af09e27fedc8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:32 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:38526
I20260926 06:56:32.492224 18027 tablet_service.cc:1514] Processing DeleteTablet for tablet 1608eefb3e704ccca93af09e27fedc8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:32 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:50454
I20260926 06:56:32.492344 17893 tablet_service.cc:1514] Processing DeleteTablet for tablet 1608eefb3e704ccca93af09e27fedc8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:32 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:59290
I20260926 06:56:32.492424 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 17830
I20260926 06:56:32.493204 18266 tablet_replica.cc:330] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a: stopping tablet replica
I20260926 06:56:32.493330 18266 raft_consensus.cc:2229] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:56:32.493495 18266 raft_consensus.cc:2258] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:56:32.493911 18266 ts_tablet_manager.cc:1905] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:56:32.495527 18266 ts_tablet_manager.cc:1918] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:56:32.495612 18266 log.cc:1196] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/1608eefb3e704ccca93af09e27fedc8f
I20260926 06:56:32.495893 18266 ts_tablet_manager.cc:1939] T 1608eefb3e704ccca93af09e27fedc8f P 1d6a9a7bdc2348d490f507f28f99e84a: Deleting consensus metadata
I20260926 06:56:32.496284 17752 catalog_manager.cc:4841] TS 1d6a9a7bdc2348d490f507f28f99e84a (127.0.191.195:33471): tablet 1608eefb3e704ccca93af09e27fedc8f (table test-table [id=9fba54a4be144c6db8646f75f5294f06]) successfully deleted
I20260926 06:56:32.497501 18267 tablet_replica.cc:330] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431: stopping tablet replica
I20260926 06:56:32.497623 18267 raft_consensus.cc:2229] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:56:32.497736 18267 raft_consensus.cc:2258] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:56:32.498172 18267 ts_tablet_manager.cc:1905] T 1608eefb3e704ccca93af09e27fedc8f P e7f139dc22d444ea89a20b9e59948431: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 06:56:32.498410 17753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:39105 (error 108)
I20260926 06:56:32.498603 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 17964
I20260926 06:56:32.499343 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.498778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39105 (local address 127.0.0.1:59306)
0926 06:56:32.499251 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:32.499287 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:39105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454}
W20260926 06:56:32.499539 17753 catalog_manager.cc:4584] TS 60aef343e7bd4afdbb1f59f823d8ce2c (127.0.191.193:39105): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1608eefb3e704ccca93af09e27fedc8f: Network error: Client connection negotiation failed: client connection to 127.0.191.193:39105: connect: Connection refused (error 111)
I20260926 06:56:32.505064 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 18098
I20260926 06:56:32.505100 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.504965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:33801 (local address 127.0.0.1:50470)
0926 06:56:32.505028 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:32.505058 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.194:33801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 06:56:32.510102 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 17736
2026-09-26T06:56:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2006 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T06:56:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:32Z Disabled control of system clock
I20260926 06:56:32.537149 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:36407
--webserver_interface=127.0.191.254
--webserver_port=0
--builtin_ntp_servers=127.0.191.212:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:36407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:56:32.616664 18273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:32.616865 18273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:32.616885 18273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:32.618048 18273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:32.618090 18273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:32.618103 18273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:32.618115 18273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:32.618127 18273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:32.619367 18273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:36407
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:36407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:32.619545 18273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:32.619764 18273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:32.624562 18279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:32.624652 18278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:32.624660 18273 server_base.cc:1034] running on GCE node
W20260926 06:56:32.624650 18281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:32.625082 18273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:32.625315 18273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:32.626468 18273 hybrid_clock.cc:648] HybridClock initialized: now 1790405792626453 us; error 36 us; skew 500 ppm
I20260926 06:56:32.627748 18273 webserver.cc:460] Webserver started at http://127.0.191.254:42599/ using document root <none> and password file <none>
I20260926 06:56:32.628021 18273 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:32.628070 18273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:32.628243 18273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:32.629277 18273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "a45ec6b20ac747b2a2584f94e0763a95"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "b6b08f0261c089ffa766cd71335c1d5e"
server_key_iv: "ac156068b6a0f992c46aff617a4695a8"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.629686 18273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "a45ec6b20ac747b2a2584f94e0763a95"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "b6b08f0261c089ffa766cd71335c1d5e"
server_key_iv: "ac156068b6a0f992c46aff617a4695a8"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.630975 18273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:32.631714 18287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.631896 18273 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:32.632006 18273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "a45ec6b20ac747b2a2584f94e0763a95"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "b6b08f0261c089ffa766cd71335c1d5e"
server_key_iv: "ac156068b6a0f992c46aff617a4695a8"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.632094 18273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:32.642508 18273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:32.642920 18273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:32.643093 18273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:32.647547 18273 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:36407
I20260926 06:56:32.647576 18339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:36407 every 8 connection(s)
I20260926 06:56:32.647953 18273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:32.649248 18340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:32.651440 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 18273
I20260926 06:56:32.651552 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:32.651731 18340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95: Bootstrap starting.
I20260926 06:56:32.651758 767 external_mini_cluster.cc:1428] Setting key 9c9aa5284beaa3d58d4ce75b19763774
I20260926 06:56:32.652432 18340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:32.652727 18340 log.cc:824] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:32.653575 18340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95: No bootstrap required, opened a new log
I20260926 06:56:32.655505 18340 raft_consensus.cc:348] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45ec6b20ac747b2a2584f94e0763a95" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36407 } }
I20260926 06:56:32.655671 18340 raft_consensus.cc:374] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:32.655702 18340 raft_consensus.cc:729] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45ec6b20ac747b2a2584f94e0763a95, State: Initialized, Role: FOLLOWER
I20260926 06:56:32.655817 18340 consensus_queue.cc:260] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45ec6b20ac747b2a2584f94e0763a95" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36407 } }
I20260926 06:56:32.655885 18340 raft_consensus.cc:388] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:32.655923 18340 raft_consensus.cc:482] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:32.655972 18340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:32.656666 18340 raft_consensus.cc:504] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45ec6b20ac747b2a2584f94e0763a95" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36407 } }
I20260926 06:56:32.656780 18340 leader_election.cc:302] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a45ec6b20ac747b2a2584f94e0763a95; no voters:
I20260926 06:56:32.656937 18340 leader_election.cc:288] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:32.656989 18345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:32.657228 18345 raft_consensus.cc:686] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [term 1 LEADER]: Becoming Leader. State: Replica: a45ec6b20ac747b2a2584f94e0763a95, State: Running, Role: LEADER
I20260926 06:56:32.657239 18340 sys_catalog.cc:564] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:32.657526 18345 consensus_queue.cc:237] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45ec6b20ac747b2a2584f94e0763a95" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36407 } }
I20260926 06:56:32.657951 18346 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a45ec6b20ac747b2a2584f94e0763a95. Latest consensus state: current_term: 1 leader_uuid: "a45ec6b20ac747b2a2584f94e0763a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45ec6b20ac747b2a2584f94e0763a95" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36407 } } }
I20260926 06:56:32.658130 18346 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:32.658008 18347 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a45ec6b20ac747b2a2584f94e0763a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45ec6b20ac747b2a2584f94e0763a95" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36407 } } }
I20260926 06:56:32.658363 18347 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:32.658679 18354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:32.658921 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.652276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41318 (local address 127.0.191.254:36407)
0926 06:56:32.652539 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:32.652549 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:32.652582 (+ 33us) server_negotiation.cc:407] Connection header received
0926 06:56:32.653107 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:32.653117 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:32.653195 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:32.653330 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:32.653621 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.654907 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.656041 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.656640 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.656773 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.657607 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:32.657611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:32.657611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:32.657849 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:32.657896 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:32.657913 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:32.658494 (+ 581us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:32.658532 (+ 38us) server_negotiation.cc:299] Negotiation successful
0926 06:56:32.658621 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":120,"threads_started":1}
I20260926 06:56:32.659158 18354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:32.660631 18354 catalog_manager.cc:1347] Generated new cluster ID: e981320b7db5491486765612bef919a1
I20260926 06:56:32.660682 18354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:32.678213 18354 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:32.678992 18354 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:32.692282 18354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95: Generated new TSK 0
I20260926 06:56:32.692492 18354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:32.699903 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:36407
--builtin_ntp_servers=127.0.191.212:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:56:32.784791 18364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:32.785025 18364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:32.785063 18364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:32.786329 18364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:32.786392 18364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:32.786440 18364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:56:32.787797 18364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:36407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:32.788056 18364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:32.788415 18364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:32.794643 18372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:32.794654 18370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:32.794910 18369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:32.795102 18364 server_base.cc:1034] running on GCE node
I20260926 06:56:32.795316 18364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:32.795571 18364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:32.796752 18364 hybrid_clock.cc:648] HybridClock initialized: now 1790405792796716 us; error 49 us; skew 500 ppm
I20260926 06:56:32.798316 18364 webserver.cc:460] Webserver started at http://127.0.191.193:35249/ using document root <none> and password file <none>
I20260926 06:56:32.798583 18364 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:32.798646 18364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:32.798774 18364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:32.800016 18364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "36949f6176bebe6634de578240f21cf0"
server_key_iv: "253f1fa1159025b6ce051fc3d6dbe9e3"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.800385 18364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "36949f6176bebe6634de578240f21cf0"
server_key_iv: "253f1fa1159025b6ce051fc3d6dbe9e3"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.801728 18364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:56:32.802521 18378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.802672 18364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.802757 18364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "36949f6176bebe6634de578240f21cf0"
server_key_iv: "253f1fa1159025b6ce051fc3d6dbe9e3"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.802809 18364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:32.811707 18364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:32.811928 18364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:32.812024 18364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:32.812182 18364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:32.812434 18364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:32.812466 18364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.812486 18364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:32.812498 18364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.818090 18364 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:46491
I20260926 06:56:32.818145 18491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:46491 every 8 connection(s)
I20260926 06:56:32.818470 18364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:56:32.819684 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 18364
I20260926 06:56:32.819802 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:56:32.820011 767 external_mini_cluster.cc:1428] Setting key 1cbeb54b5c94944c1ef47da86ad836da
I20260926 06:56:32.821070 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:36407
--builtin_ntp_servers=127.0.191.212:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 06:56:32.824115 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.819147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:59595 (local address 127.0.191.254:36407)
0926 06:56:32.819188 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:32.819190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:32.819393 (+ 203us) server_negotiation.cc:407] Connection header received
0926 06:56:32.819949 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:32.819950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:32.819963 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:32.820025 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:32.820885 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.821682 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.822661 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.822877 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.822964 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.823841 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:32.823843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:32.823843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:32.823848 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:32.823868 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:32.823872 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:32.823930 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:32.824012 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 06:56:32.824038 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:56:32.824389 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.819105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36407 (local address 127.0.191.193:59595)
0926 06:56:32.819274 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:32.819306 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:32.819849 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:32.820097 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:32.820101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:32.820111 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:32.820741 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.820780 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.821770 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.821771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.822546 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.822548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.822975 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.822976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.823118 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.823692 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:56:32.823693 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:32.823719 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:56:32.823740 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:32.823932 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:32.823938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:32.823941 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:56:32.824015 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 06:56:32.824114 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
I20260926 06:56:32.824970 18492 heartbeater.cc:347] Connected to a master server at 127.0.191.254:36407
I20260926 06:56:32.825047 18492 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:32.825241 18492 heartbeater.cc:511] Master 127.0.191.254:36407 requested a full tablet report, sending...
I20260926 06:56:32.825778 18303 ts_manager.cc:194] Registered new tserver with Master: a6cdb2d01c2c4e198a6d8eb93fb3f6a0 (127.0.191.193:46491)
I20260926 06:56:32.826761 18303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.191.193:59595
W20260926 06:56:32.901166 18495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:32.901364 18495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:32.901403 18495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:32.902532 18495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:32.902580 18495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:32.902604 18495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:56:32.904232 18495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:36407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:32.904579 18495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:32.904857 18495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:32.910594 18500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:32.910619 18503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:32.910651 18501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:32.910619 18495 server_base.cc:1034] running on GCE node
I20260926 06:56:32.911029 18495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:32.911233 18495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:32.912425 18495 hybrid_clock.cc:648] HybridClock initialized: now 1790405792912488 us; error 108 us; skew 500 ppm
I20260926 06:56:32.913779 18495 webserver.cc:460] Webserver started at http://127.0.191.194:38645/ using document root <none> and password file <none>
I20260926 06:56:32.913992 18495 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:32.914036 18495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:32.914124 18495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:32.915155 18495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "934b8c4512ce45419ce38641b270f2e0"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "731a99218ee9c88b91f737c3dd1237f9"
server_key_iv: "2337e2cb404d2406ffb03234cae713fb"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.915459 18495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "934b8c4512ce45419ce38641b270f2e0"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "731a99218ee9c88b91f737c3dd1237f9"
server_key_iv: "2337e2cb404d2406ffb03234cae713fb"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.916662 18495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:56:32.917317 18509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.917543 18495 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:32.917626 18495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "934b8c4512ce45419ce38641b270f2e0"
format_stamp: "Formatted at 2026-09-26 06:56:32 on dist-test-slave-ws74"
server_key: "731a99218ee9c88b91f737c3dd1237f9"
server_key_iv: "2337e2cb404d2406ffb03234cae713fb"
server_key_version: "encryptionkey@0"
I20260926 06:56:32.917680 18495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:32.932201 18495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:32.932459 18495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:32.932562 18495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:32.932744 18495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:32.933100 18495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:32.933139 18495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.933161 18495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:32.933208 18495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:32.939262 18495 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:37619
I20260926 06:56:32.939364 18622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:37619 every 8 connection(s)
I20260926 06:56:32.939653 18495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:56:32.945535 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.940428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:48305 (local address 127.0.191.254:36407)
0926 06:56:32.940477 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:32.940480 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:32.940699 (+ 219us) server_negotiation.cc:407] Connection header received
0926 06:56:32.941283 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:32.941284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:32.941311 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:32.941391 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:32.942150 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.942704 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.943967 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.944198 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.944283 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.945171 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:32.945172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:32.945173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:32.945178 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:32.945193 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:32.945203 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:32.945267 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:32.945426 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 06:56:32.945453 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:56:32.945775 18625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:32.940373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36407 (local address 127.0.191.194:48305)
0926 06:56:32.940568 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:32.940604 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:32.941168 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:32.941463 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:32.941467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:32.941479 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:32.942034 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.942039 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.942817 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.942818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.943853 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:32.943856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:32.944298 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:32.944299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:32.944445 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:32.944996 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:56:32.944997 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:32.945026 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:56:32.945047 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:32.945267 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:32.945274 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:32.945277 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:56:32.945427 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 06:56:32.945528 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260926 06:56:32.946038 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 18495
I20260926 06:56:32.946161 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:56:32.946487 18623 heartbeater.cc:347] Connected to a master server at 127.0.191.254:36407
I20260926 06:56:32.946488 767 external_mini_cluster.cc:1428] Setting key 5930b30ba4c3e2a1bbdd1de9f7381dd3
I20260926 06:56:32.946599 18623 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:32.946792 18623 heartbeater.cc:511] Master 127.0.191.254:36407 requested a full tablet report, sending...
I20260926 06:56:32.947192 18303 ts_manager.cc:194] Registered new tserver with Master: 934b8c4512ce45419ce38641b270f2e0 (127.0.191.194:37619)
I20260926 06:56:32.947605 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:36407
--builtin_ntp_servers=127.0.191.212:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 06:56:32.947755 18303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.191.194:48305
W20260926 06:56:33.031878 18626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:33.032083 18626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:33.032104 18626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:33.033442 18626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:33.033488 18626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:33.033517 18626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:56:33.034988 18626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:36407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:33.035229 18626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:33.035562 18626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:33.040553 18631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:33.040572 18634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:33.040562 18632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:33.040788 18626 server_base.cc:1034] running on GCE node
I20260926 06:56:33.040993 18626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:33.041182 18626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:33.042318 18626 hybrid_clock.cc:648] HybridClock initialized: now 1790405793042279 us; error 54 us; skew 500 ppm
I20260926 06:56:33.043574 18626 webserver.cc:460] Webserver started at http://127.0.191.195:35239/ using document root <none> and password file <none>
I20260926 06:56:33.043803 18626 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:33.043848 18626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:33.044020 18626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:33.045125 18626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "5a98e4a9ff5042a790cda8775a45cbc1"
format_stamp: "Formatted at 2026-09-26 06:56:33 on dist-test-slave-ws74"
server_key: "acb2ec5aa0612fc1bbe8e6dec6d90d1f"
server_key_iv: "352179ffe2dba24c22f76ba96186abff"
server_key_version: "encryptionkey@0"
I20260926 06:56:33.045504 18626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "5a98e4a9ff5042a790cda8775a45cbc1"
format_stamp: "Formatted at 2026-09-26 06:56:33 on dist-test-slave-ws74"
server_key: "acb2ec5aa0612fc1bbe8e6dec6d90d1f"
server_key_iv: "352179ffe2dba24c22f76ba96186abff"
server_key_version: "encryptionkey@0"
I20260926 06:56:33.046785 18626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:33.047487 18640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:33.047641 18626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:33.047735 18626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "5a98e4a9ff5042a790cda8775a45cbc1"
format_stamp: "Formatted at 2026-09-26 06:56:33 on dist-test-slave-ws74"
server_key: "acb2ec5aa0612fc1bbe8e6dec6d90d1f"
server_key_iv: "352179ffe2dba24c22f76ba96186abff"
server_key_version: "encryptionkey@0"
I20260926 06:56:33.047812 18626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:33.058399 18626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:33.058681 18626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:33.058869 18626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:33.059087 18626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:33.059399 18626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:33.059432 18626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:33.059453 18626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:33.059510 18626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:33.065114 18626 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:35843
I20260926 06:56:33.065182 18753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:35843 every 8 connection(s)
I20260926 06:56:33.065572 18626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:56:33.072324 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 18626
I20260926 06:56:33.072451 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:56:33.072677 767 external_mini_cluster.cc:1428] Setting key 8698c6708a4b05eb91c2ccf4ecf32735
I20260926 06:56:33.074958 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.066350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:35431 (local address 127.0.191.254:36407)
0926 06:56:33.066418 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:33.066421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:33.069665 (+ 3244us) server_negotiation.cc:407] Connection header received
0926 06:56:33.070195 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:33.070197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:33.070210 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:33.070272 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:33.070947 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.071494 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.072903 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.073242 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.073322 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.074588 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:33.074590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:33.074590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:33.074595 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:33.074613 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:33.074617 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:33.074700 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:33.074852 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 06:56:33.074879 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:56:33.075165 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.066287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:36407 (local address 127.0.191.195:35431)
0926 06:56:33.069534 (+ 3247us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:33.069568 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:33.070087 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:33.070347 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:33.070351 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:33.070358 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:33.070833 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:33.070837 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.071582 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.071583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:33.072768 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:33.072781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.073326 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.073327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:33.073512 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.074246 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:56:33.074247 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:33.074311 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:56:33.074340 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:33.074733 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:33.074739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:33.074742 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:56:33.074840 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 06:56:33.074947 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3186,"thread_start_us":81,"threads_started":1}
I20260926 06:56:33.075637 18754 heartbeater.cc:347] Connected to a master server at 127.0.191.254:36407
I20260926 06:56:33.075750 18754 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:33.075937 18754 heartbeater.cc:511] Master 127.0.191.254:36407 requested a full tablet report, sending...
I20260926 06:56:33.076395 18303 ts_manager.cc:194] Registered new tserver with Master: 5a98e4a9ff5042a790cda8775a45cbc1 (127.0.191.195:35843)
I20260926 06:56:33.076725 18303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.191.195:35431
I20260926 06:56:33.084635 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:56:33.088941 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.086022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41322 (local address 127.0.191.254:36407)
0926 06:56:33.086058 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:33.086060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:33.086184 (+ 124us) server_negotiation.cc:407] Connection header received
0926 06:56:33.086258 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:33.086258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:33.086270 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:33.086324 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:33.086495 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.086970 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.087715 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.087927 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.088010 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.088709 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:33.088710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:33.088710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:33.088715 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:33.088728 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:33.088732 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:33.088779 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:33.088839 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 06:56:33.088859 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 06:56:33.090567 18303 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41322:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:56:33.092012 18303 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:56:33.098201 18762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.094239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:46491 (local address 127.0.0.1:58236)
0926 06:56:33.094389 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:33.094428 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:33.094489 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:33.094731 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:33.094733 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:33.094736 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:33.094797 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:33.094799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.095811 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.095813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:33.097101 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:33.097103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.097920 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.097921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:33.097993 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.098006 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 06:56:33.098081 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:56:33.098132 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":53,"threads_started":1}
I20260926 06:56:33.098212 18764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.094329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58236 (local address 127.0.191.193:46491)
0926 06:56:33.094545 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:33.094550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:33.094559 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:56:33.094600 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:33.094602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:33.094619 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:33.094879 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:33.094941 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.095731 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.097181 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.097814 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.097916 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.098008 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:33.098094 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 06:56:33.098132 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":93,"threads_started":1}
I20260926 06:56:33.098416 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.094276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:37619 (local address 127.0.0.1:34256)
0926 06:56:33.094613 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:33.094622 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:33.094668 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:33.095069 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:33.095070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:33.095106 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:33.095144 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:33.095146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.096416 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.096417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:33.097132 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:33.097133 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.098165 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.098167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:33.098232 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.098236 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:56:33.098285 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 06:56:33.098330 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":2,"thread_start_us":123,"threads_started":1}
I20260926 06:56:33.098693 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.094373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34256 (local address 127.0.191.194:37619)
0926 06:56:33.094562 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:33.094566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:33.094731 (+ 165us) server_negotiation.cc:407] Connection header received
0926 06:56:33.094917 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:33.094920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:33.094938 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:33.095010 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:33.095237 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.096329 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.097222 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.097971 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.098045 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.098129 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:33.098564 (+ 435us) server_negotiation.cc:299] Negotiation successful
0926 06:56:33.098614 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":78,"threads_started":1}
I20260926 06:56:33.098961 18426 tablet_service.cc:1467] Processing CreateTablet for tablet 3a366bd6ae444355b76ba3402b31e466 (DEFAULT_TABLE table=test-table [id=e30de64a994542529b6d27f77bc4243f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:33.098990 18763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.094506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:35843 (local address 127.0.0.1:54852)
0926 06:56:33.094628 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:33.094637 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:33.094689 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:33.095009 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:33.095010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:33.095012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:33.095056 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:33.095057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.096392 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.096393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:33.097454 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:33.097456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.098668 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.098669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:33.098729 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.098808 (+ 79us) client_negotiation.cc:769] Sending connection context
0926 06:56:33.098875 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 06:56:33.098914 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":6,"thread_start_us":66,"threads_started":1}
I20260926 06:56:33.099352 18426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a366bd6ae444355b76ba3402b31e466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:33.099403 18767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.094668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.0.191.195:35843)
0926 06:56:33.094807 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:33.094810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:33.094830 (+ 20us) server_negotiation.cc:407] Connection header received
0926 06:56:33.094887 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:33.094889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:33.094904 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:33.095139 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:33.095202 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.096298 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.097533 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.098195 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.098651 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.098738 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:33.099280 (+ 542us) server_negotiation.cc:299] Negotiation successful
0926 06:56:33.099319 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260926 06:56:33.099351 18553 tablet_service.cc:1467] Processing CreateTablet for tablet 3a366bd6ae444355b76ba3402b31e466 (DEFAULT_TABLE table=test-table [id=e30de64a994542529b6d27f77bc4243f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:33.099676 18553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a366bd6ae444355b76ba3402b31e466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:33.101908 18768 tablet_bootstrap.cc:492] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0: Bootstrap starting.
I20260926 06:56:33.102440 18769 tablet_bootstrap.cc:492] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0: Bootstrap starting.
I20260926 06:56:33.102542 18768 tablet_bootstrap.cc:654] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:33.102779 18688 tablet_service.cc:1467] Processing CreateTablet for tablet 3a366bd6ae444355b76ba3402b31e466 (DEFAULT_TABLE table=test-table [id=e30de64a994542529b6d27f77bc4243f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:33.102886 18768 log.cc:824] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:33.103116 18688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a366bd6ae444355b76ba3402b31e466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:33.103154 18769 tablet_bootstrap.cc:654] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:33.103494 18769 log.cc:824] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:33.103683 18768 tablet_bootstrap.cc:492] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0: No bootstrap required, opened a new log
I20260926 06:56:33.103758 18768 ts_tablet_manager.cc:1397] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:56:33.104305 18769 tablet_bootstrap.cc:492] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0: No bootstrap required, opened a new log
I20260926 06:56:33.104395 18769 ts_tablet_manager.cc:1397] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:56:33.106089 18772 tablet_bootstrap.cc:492] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1: Bootstrap starting.
I20260926 06:56:33.106448 18768 raft_consensus.cc:348] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 46491 } } peers { permanent_uuid: "934b8c4512ce45419ce38641b270f2e0" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37619 } } peers { permanent_uuid: "5a98e4a9ff5042a790cda8775a45cbc1" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35843 } }
I20260926 06:56:33.106618 18768 raft_consensus.cc:374] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:33.106657 18768 raft_consensus.cc:729] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6cdb2d01c2c4e198a6d8eb93fb3f6a0, State: Initialized, Role: FOLLOWER
I20260926 06:56:33.106799 18772 tablet_bootstrap.cc:654] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:33.106772 18768 consensus_queue.cc:260] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 46491 } } peers { permanent_uuid: "934b8c4512ce45419ce38641b270f2e0" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37619 } } peers { permanent_uuid: "5a98e4a9ff5042a790cda8775a45cbc1" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35843 } }
I20260926 06:56:33.107026 18768 ts_tablet_manager.cc:1428] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260926 06:56:33.106930 18769 raft_consensus.cc:348] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 46491 } } peers { permanent_uuid: "934b8c4512ce45419ce38641b270f2e0" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37619 } } peers { permanent_uuid: "5a98e4a9ff5042a790cda8775a45cbc1" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35843 } }
I20260926 06:56:33.107072 18769 raft_consensus.cc:374] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:33.107105 18769 raft_consensus.cc:729] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 934b8c4512ce45419ce38641b270f2e0, State: Initialized, Role: FOLLOWER
I20260926 06:56:33.107148 18492 heartbeater.cc:503] Master 127.0.191.254:36407 was elected leader, sending a full tablet report...
I20260926 06:56:33.107131 18772 log.cc:824] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:33.107246 18769 consensus_queue.cc:260] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 46491 } } peers { permanent_uuid: "934b8c4512ce45419ce38641b270f2e0" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37619 } } peers { permanent_uuid: "5a98e4a9ff5042a790cda8775a45cbc1" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35843 } }
I20260926 06:56:33.107533 18769 ts_tablet_manager.cc:1428] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:56:33.107653 18623 heartbeater.cc:503] Master 127.0.191.254:36407 was elected leader, sending a full tablet report...
I20260926 06:56:33.107947 18772 tablet_bootstrap.cc:492] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1: No bootstrap required, opened a new log
I20260926 06:56:33.108011 18772 ts_tablet_manager.cc:1397] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:56:33.109974 18772 raft_consensus.cc:348] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 46491 } } peers { permanent_uuid: "934b8c4512ce45419ce38641b270f2e0" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37619 } } peers { permanent_uuid: "5a98e4a9ff5042a790cda8775a45cbc1" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35843 } }
I20260926 06:56:33.110108 18772 raft_consensus.cc:374] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:33.110152 18772 raft_consensus.cc:729] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a98e4a9ff5042a790cda8775a45cbc1, State: Initialized, Role: FOLLOWER
I20260926 06:56:33.110255 18772 consensus_queue.cc:260] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 46491 } } peers { permanent_uuid: "934b8c4512ce45419ce38641b270f2e0" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37619 } } peers { permanent_uuid: "5a98e4a9ff5042a790cda8775a45cbc1" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35843 } }
I20260926 06:56:33.110498 18772 ts_tablet_manager.cc:1428] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 06:56:33.110585 18754 heartbeater.cc:503] Master 127.0.191.254:36407 was elected leader, sending a full tablet report...
I20260926 06:56:33.185187 18773 raft_consensus.cc:482] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:56:33.185317 18773 raft_consensus.cc:504] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 46491 } } peers { permanent_uuid: "934b8c4512ce45419ce38641b270f2e0" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37619 } } peers { permanent_uuid: "5a98e4a9ff5042a790cda8775a45cbc1" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35843 } }
I20260926 06:56:33.185746 18773 leader_election.cc:288] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 934b8c4512ce45419ce38641b270f2e0 (127.0.191.194:37619), 5a98e4a9ff5042a790cda8775a45cbc1 (127.0.191.195:35843)
I20260926 06:56:33.189623 18767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.186277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:48087 (local address 127.0.191.195:35843)
0926 06:56:33.186327 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:33.186329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:33.186449 (+ 120us) server_negotiation.cc:407] Connection header received
0926 06:56:33.186500 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:33.186501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:33.186511 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:33.186570 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:33.186752 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.187429 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.188676 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.189212 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.189291 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.189297 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:33.189515 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 06:56:33.189553 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 06:56:33.189623 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.186113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:52611 (local address 127.0.191.194:37619)
0926 06:56:33.186227 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:33.186229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:33.186239 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:56:33.186271 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:33.186272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:33.186282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:33.186349 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:33.186517 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.187242 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.188651 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.189212 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.189290 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.189308 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:33.189514 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 06:56:33.189553 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260926 06:56:33.189646 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.186032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:37619 (local address 127.0.191.193:52611)
0926 06:56:33.186105 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:33.186118 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:33.186178 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:33.186367 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:33.186368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:33.186369 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:33.186444 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:33.186446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.187330 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.187331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:33.188537 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:33.188539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.189315 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.189316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:33.189426 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.189430 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:56:33.189500 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 06:56:33.189558 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":11}
I20260926 06:56:33.189651 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.186162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:35843 (local address 127.0.191.193:48087)
0926 06:56:33.186370 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:33.186381 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:33.186436 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:33.186601 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:33.186602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:33.186605 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:33.186663 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:33.186664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.187506 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.187507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:33.188598 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:33.188600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.189315 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.189316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:33.189446 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.189449 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:56:33.189503 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 06:56:33.189557 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":3,"thread_start_us":103,"threads_started":1}
I20260926 06:56:33.190034 18577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a366bd6ae444355b76ba3402b31e466" candidate_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "934b8c4512ce45419ce38641b270f2e0" is_pre_election: true
W20260926 06:56:33.190161 18624 tablet.cc:2369] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:56:33.190191 18577 raft_consensus.cc:2454] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6cdb2d01c2c4e198a6d8eb93fb3f6a0 in term 0.
I20260926 06:56:33.190253 18708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a366bd6ae444355b76ba3402b31e466" candidate_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a98e4a9ff5042a790cda8775a45cbc1" is_pre_election: true
I20260926 06:56:33.190416 18708 raft_consensus.cc:2454] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6cdb2d01c2c4e198a6d8eb93fb3f6a0 in term 0.
I20260926 06:56:33.190465 18379 leader_election.cc:302] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 934b8c4512ce45419ce38641b270f2e0, a6cdb2d01c2c4e198a6d8eb93fb3f6a0; no voters:
I20260926 06:56:33.190588 18773 raft_consensus.cc:2781] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:56:33.190644 18773 raft_consensus.cc:482] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:56:33.190666 18773 raft_consensus.cc:3037] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:33.191324 18773 raft_consensus.cc:504] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 46491 } } peers { permanent_uuid: "934b8c4512ce45419ce38641b270f2e0" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37619 } } peers { permanent_uuid: "5a98e4a9ff5042a790cda8775a45cbc1" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35843 } }
I20260926 06:56:33.191459 18773 leader_election.cc:288] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [CANDIDATE]: Term 1 election: Requested vote from peers 934b8c4512ce45419ce38641b270f2e0 (127.0.191.194:37619), 5a98e4a9ff5042a790cda8775a45cbc1 (127.0.191.195:35843)
I20260926 06:56:33.191612 18577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a366bd6ae444355b76ba3402b31e466" candidate_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "934b8c4512ce45419ce38641b270f2e0"
I20260926 06:56:33.191643 18708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a366bd6ae444355b76ba3402b31e466" candidate_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a98e4a9ff5042a790cda8775a45cbc1"
I20260926 06:56:33.191720 18708 raft_consensus.cc:3037] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:33.191722 18577 raft_consensus.cc:3037] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:33.192406 18708 raft_consensus.cc:2454] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cdb2d01c2c4e198a6d8eb93fb3f6a0 in term 1.
I20260926 06:56:33.192525 18577 raft_consensus.cc:2454] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cdb2d01c2c4e198a6d8eb93fb3f6a0 in term 1.
I20260926 06:56:33.192593 18379 leader_election.cc:302] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a98e4a9ff5042a790cda8775a45cbc1, a6cdb2d01c2c4e198a6d8eb93fb3f6a0; no voters:
I20260926 06:56:33.192698 18773 raft_consensus.cc:2781] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:33.192844 18773 raft_consensus.cc:686] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [term 1 LEADER]: Becoming Leader. State: Replica: a6cdb2d01c2c4e198a6d8eb93fb3f6a0, State: Running, Role: LEADER
I20260926 06:56:33.192917 18773 consensus_queue.cc:237] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 46491 } } peers { permanent_uuid: "934b8c4512ce45419ce38641b270f2e0" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37619 } } peers { permanent_uuid: "5a98e4a9ff5042a790cda8775a45cbc1" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35843 } }
I20260926 06:56:33.193739 18303 catalog_manager.cc:5495] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6cdb2d01c2c4e198a6d8eb93fb3f6a0 (127.0.191.193). New cstate: current_term: 1 leader_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cdb2d01c2c4e198a6d8eb93fb3f6a0" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 46491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "934b8c4512ce45419ce38641b270f2e0" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a98e4a9ff5042a790cda8775a45cbc1" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35843 } health_report { overall_health: UNKNOWN } } }
I20260926 06:56:33.213423 18764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.211021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58252 (local address 127.0.191.193:46491)
0926 06:56:33.211085 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:33.211088 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:33.211112 (+ 24us) server_negotiation.cc:407] Connection header received
0926 06:56:33.211147 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:33.211148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:33.211169 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:33.211263 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:56:33.211417 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.211877 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.212615 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:33.212819 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:33.212893 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:33.213048 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:56:33.213150 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:56:33.213233 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:33.213275 (+ 42us) server_negotiation.cc:299] Negotiation successful
0926 06:56:33.213295 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 06:56:33.215412 18708 raft_consensus.cc:1261] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1 [term 1 FOLLOWER]: Refusing update from remote peer a6cdb2d01c2c4e198a6d8eb93fb3f6a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:56:33.215425 18577 raft_consensus.cc:1261] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0 [term 1 FOLLOWER]: Refusing update from remote peer a6cdb2d01c2c4e198a6d8eb93fb3f6a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:56:33.215824 18778 consensus_queue.cc:1035] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a98e4a9ff5042a790cda8775a45cbc1" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 35843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:56:33.215945 18773 consensus_queue.cc:1035] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "934b8c4512ce45419ce38641b270f2e0" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 37619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:56:33.218343 18783 mvcc.cc:205] Tried to move back new op lower bound from 7333502129006616576 to 7333502128918622208. Current Snapshot: MvccSnapshot[applied={T|T < 7333502129006616576}]
I20260926 06:56:33.218976 18782 mvcc.cc:205] Tried to move back new op lower bound from 7333502129006616576 to 7333502128918622208. Current Snapshot: MvccSnapshot[applied={T|T < 7333502129006616576}]
I20260926 06:56:33.221112 18303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:56:33.221201 18303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:56:33.222059 18303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a45ec6b20ac747b2a2584f94e0763a95: Sending DeleteTablet for 3 replicas of tablet 3a366bd6ae444355b76ba3402b31e466
I20260926 06:56:33.222409 18426 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a366bd6ae444355b76ba3402b31e466 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:33 UTC) from {username='slave'} at 127.0.0.1:58236
I20260926 06:56:33.222651 18791 tablet_replica.cc:330] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0: stopping tablet replica
I20260926 06:56:33.222625 18688 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a366bd6ae444355b76ba3402b31e466 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:33 UTC) from {username='slave'} at 127.0.0.1:54852
I20260926 06:56:33.222832 18791 raft_consensus.cc:2229] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:56:33.222852 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 18364
I20260926 06:56:33.222860 18553 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a366bd6ae444355b76ba3402b31e466 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:33 UTC) from {username='slave'} at 127.0.0.1:34256
I20260926 06:56:33.223061 18793 tablet_replica.cc:330] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0: stopping tablet replica
I20260926 06:56:33.223065 18791 raft_consensus.cc:2258] T 3a366bd6ae444355b76ba3402b31e466 P a6cdb2d01c2c4e198a6d8eb93fb3f6a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:56:33.223155 18793 raft_consensus.cc:2229] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:56:33.223454 18792 tablet_replica.cc:330] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1: stopping tablet replica
I20260926 06:56:33.223565 18792 raft_consensus.cc:2229] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:56:33.223663 18792 raft_consensus.cc:2258] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:56:33.223834 18793 raft_consensus.cc:2258] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:56:33.224174 18792 ts_tablet_manager.cc:1905] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:56:33.224196 18793 ts_tablet_manager.cc:1905] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:56:33.225912 18793 ts_tablet_manager.cc:1918] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:56:33.225912 18792 ts_tablet_manager.cc:1918] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:56:33.225989 18792 log.cc:1196] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/3a366bd6ae444355b76ba3402b31e466
I20260926 06:56:33.225989 18793 log.cc:1196] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/3a366bd6ae444355b76ba3402b31e466
I20260926 06:56:33.226253 18793 ts_tablet_manager.cc:1939] T 3a366bd6ae444355b76ba3402b31e466 P 934b8c4512ce45419ce38641b270f2e0: Deleting consensus metadata
I20260926 06:56:33.226253 18792 ts_tablet_manager.cc:1939] T 3a366bd6ae444355b76ba3402b31e466 P 5a98e4a9ff5042a790cda8775a45cbc1: Deleting consensus metadata
I20260926 06:56:33.226673 18288 catalog_manager.cc:4841] TS 934b8c4512ce45419ce38641b270f2e0 (127.0.191.194:37619): tablet 3a366bd6ae444355b76ba3402b31e466 (table test-table [id=e30de64a994542529b6d27f77bc4243f]) successfully deleted
I20260926 06:56:33.226768 18288 catalog_manager.cc:4841] TS 5a98e4a9ff5042a790cda8775a45cbc1 (127.0.191.195:35843): tablet 3a366bd6ae444355b76ba3402b31e466 (table test-table [id=e30de64a994542529b6d27f77bc4243f]) successfully deleted
W20260926 06:56:33.229225 18291 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:46491 (error 108)
I20260926 06:56:33.229694 18762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:33.229560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:46491 (local address 127.0.0.1:58264)
0926 06:56:33.229618 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:33.229656 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:46491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 06:56:33.229795 18291 catalog_manager.cc:4584] TS a6cdb2d01c2c4e198a6d8eb93fb3f6a0 (127.0.191.193:46491): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3a366bd6ae444355b76ba3402b31e466: Network error: Client connection negotiation failed: client connection to 127.0.191.193:46491: connect: Connection refused (error 111)
I20260926 06:56:33.229827 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 18495
I20260926 06:56:33.234510 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 18626
I20260926 06:56:33.239249 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 18273
2026-09-26T06:56:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (727 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:33 dist-test-slave-ws74 krb5kdc[18797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:33 dist-test-slave-ws74 krb5kdc[18797](info): set up 2 sockets
Sep 26 06:56:33 dist-test-slave-ws74 krb5kdc[18797](info): commencing operation
krb5kdc: starting...
W20260926 06:56:34.285161 767 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:34 dist-test-slave-ws74 krb5kdc[18797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:56:34.404548 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:40959
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:35859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:40959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:34.483105 18810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:34.483359 18810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:34.483398 18810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:34.484652 18810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:34.484730 18810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:34.484754 18810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:34.484789 18810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:34.484835 18810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:34.486317 18810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:35859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:40959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:40959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:34.486588 18810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:34.486814 18810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:34.491605 18815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:34.491605 18816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:34.491627 18818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:34.491748 18810 server_base.cc:1034] running on GCE node
I20260926 06:56:34.492050 18810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:34.492419 18810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:34.493609 18810 hybrid_clock.cc:648] HybridClock initialized: now 1790405794493584 us; error 40 us; skew 500 ppm
Sep 26 06:56:34 dist-test-slave-ws74 krb5kdc[18797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:34.496042 18810 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:56:34.496477 18810 webserver.cc:460] Webserver started at http://127.0.191.254:45383/ using document root <none> and password file <none>
I20260926 06:56:34.496696 18810 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:34.496764 18810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:34.496923 18810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:34.498018 18810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "40691525a2bc41d9b4b3ffe02c9aeded"
format_stamp: "Formatted at 2026-09-26 06:56:34 on dist-test-slave-ws74"
server_key: "64b589f47210dbd9042b747af5aa883d"
server_key_iv: "d82a1151833f5044d9368edf9c3c0c23"
server_key_version: "encryptionkey@0"
I20260926 06:56:34.498364 18810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "40691525a2bc41d9b4b3ffe02c9aeded"
format_stamp: "Formatted at 2026-09-26 06:56:34 on dist-test-slave-ws74"
server_key: "64b589f47210dbd9042b747af5aa883d"
server_key_iv: "d82a1151833f5044d9368edf9c3c0c23"
server_key_version: "encryptionkey@0"
I20260926 06:56:34.499661 18810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:56:34.500386 18825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:34.500562 18810 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:34.500664 18810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "40691525a2bc41d9b4b3ffe02c9aeded"
format_stamp: "Formatted at 2026-09-26 06:56:34 on dist-test-slave-ws74"
server_key: "64b589f47210dbd9042b747af5aa883d"
server_key_iv: "d82a1151833f5044d9368edf9c3c0c23"
server_key_version: "encryptionkey@0"
I20260926 06:56:34.500744 18810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:34.529820 18810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:34.531770 18810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:34.531939 18810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:34.535954 18810 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:40959
I20260926 06:56:34.535995 18877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:40959 every 8 connection(s)
I20260926 06:56:34.536355 18810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:34.537246 18878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:34.539172 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 18810
I20260926 06:56:34.539283 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:34.539520 767 external_mini_cluster.cc:1428] Setting key 4e9fa3de583af1f32e015e50df80a217
I20260926 06:56:34.539785 18878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded: Bootstrap starting.
I20260926 06:56:34.540498 18878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:34.540787 18878 log.cc:824] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:34.541582 18878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded: No bootstrap required, opened a new log
I20260926 06:56:34.543800 18878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40691525a2bc41d9b4b3ffe02c9aeded" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40959 } }
I20260926 06:56:34.543974 18878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:34.544019 18878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40691525a2bc41d9b4b3ffe02c9aeded, State: Initialized, Role: FOLLOWER
I20260926 06:56:34.544137 18878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40691525a2bc41d9b4b3ffe02c9aeded" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40959 } }
I20260926 06:56:34.544209 18878 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:34.544237 18878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:34.544281 18878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:34.544942 18878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40691525a2bc41d9b4b3ffe02c9aeded" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40959 } }
I20260926 06:56:34.545063 18878 leader_election.cc:302] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40691525a2bc41d9b4b3ffe02c9aeded; no voters:
I20260926 06:56:34.545229 18878 leader_election.cc:288] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:34.545271 18883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:34.545403 18883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [term 1 LEADER]: Becoming Leader. State: Replica: 40691525a2bc41d9b4b3ffe02c9aeded, State: Running, Role: LEADER
I20260926 06:56:34.545504 18878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:34.545481 18883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40691525a2bc41d9b4b3ffe02c9aeded" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40959 } }
I20260926 06:56:34.545802 18883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40691525a2bc41d9b4b3ffe02c9aeded. Latest consensus state: current_term: 1 leader_uuid: "40691525a2bc41d9b4b3ffe02c9aeded" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40691525a2bc41d9b4b3ffe02c9aeded" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40959 } } }
I20260926 06:56:34.545841 18885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40691525a2bc41d9b4b3ffe02c9aeded" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40691525a2bc41d9b4b3ffe02c9aeded" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40959 } } }
I20260926 06:56:34.545873 18883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:34.546123 18885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:34.546411 18892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 26 06:56:34 dist-test-slave-ws74 krb5kdc[18797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:34.547140 18892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:34.549482 18892 catalog_manager.cc:1347] Generated new cluster ID: 36fe7c8029904e2e96162a9fd0195583
I20260926 06:56:34.549537 18892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:34.551154 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:34.540029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51348 (local address 127.0.191.254:40959)
0926 06:56:34.540206 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:34.540214 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:34.540240 (+ 26us) server_negotiation.cc:407] Connection header received
0926 06:56:34.540706 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:34.540713 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:34.540761 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:34.540938 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:34.541759 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:34.543131 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:34.543921 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:34.544538 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:34.544650 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:34.547698 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:34.547702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:34.547702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:34.547713 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:34.549358 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:34.549792 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:34.549795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:34.549795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:34.549885 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:34.550048 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:34.550049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:34.550049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:34.550304 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:34.550439 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:34.550834 (+ 395us) server_negotiation.cc:299] Negotiation successful
0926 06:56:34.550954 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260926 06:56:34.559677 18892 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:34.560217 18892 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:34.569008 18892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded: Generated new TSK 0
I20260926 06:56:34.569221 18892 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:56:34.600100 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:40959
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:35859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:34.699452 18904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:34.699656 18904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:34.699678 18904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:34.701166 18904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:34.701218 18904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:34.701246 18904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:56:34.702883 18904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:35859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:40959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:34.703123 18904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:34.703399 18904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:34.708455 18909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:34.708458 18912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:34.708455 18910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:34.708678 18904 server_base.cc:1034] running on GCE node
I20260926 06:56:34.708935 18904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:34.709172 18904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:34.710318 18904 hybrid_clock.cc:648] HybridClock initialized: now 1790405794710299 us; error 42 us; skew 500 ppm
Sep 26 06:56:34 dist-test-slave-ws74 krb5kdc[18797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:34.712904 18904 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:56:34.713342 18904 webserver.cc:460] Webserver started at http://127.0.191.193:39569/ using document root <none> and password file <none>
I20260926 06:56:34.713587 18904 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:34.713632 18904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:34.713719 18904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:34.714687 18904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "e3e732dc052346e9a008cafad975a5da"
format_stamp: "Formatted at 2026-09-26 06:56:34 on dist-test-slave-ws74"
server_key: "09d71d005eb37674a05b8fd7bdfb44dd"
server_key_iv: "302f5dfa7cc93dd210c50eca2be3c70a"
server_key_version: "encryptionkey@0"
I20260926 06:56:34.714993 18904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "e3e732dc052346e9a008cafad975a5da"
format_stamp: "Formatted at 2026-09-26 06:56:34 on dist-test-slave-ws74"
server_key: "09d71d005eb37674a05b8fd7bdfb44dd"
server_key_iv: "302f5dfa7cc93dd210c50eca2be3c70a"
server_key_version: "encryptionkey@0"
I20260926 06:56:34.716223 18904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:56:34.716899 18919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:34.717068 18904 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:56:34.717163 18904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "e3e732dc052346e9a008cafad975a5da"
format_stamp: "Formatted at 2026-09-26 06:56:34 on dist-test-slave-ws74"
server_key: "09d71d005eb37674a05b8fd7bdfb44dd"
server_key_iv: "302f5dfa7cc93dd210c50eca2be3c70a"
server_key_version: "encryptionkey@0"
I20260926 06:56:34.717243 18904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:34.733469 18904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:34.735813 18904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:34.735998 18904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:34.736244 18904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:34.736548 18904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:34.736603 18904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:34.736642 18904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:34.736670 18904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:34.742419 18904 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:36419
I20260926 06:56:34.742467 19032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:36419 every 8 connection(s)
I20260926 06:56:34.742812 18904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:56:34.746099 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 18904
I20260926 06:56:34.746219 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:56:34.746456 767 external_mini_cluster.cc:1428] Setting key 23fd372a74995c5e8a71a5fd97d16ef7
Sep 26 06:56:34 dist-test-slave-ws74 krb5kdc[18797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:34.753026 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:34.743562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:54185 (local address 127.0.191.254:40959)
0926 06:56:34.743632 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:34.743634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:34.743840 (+ 206us) server_negotiation.cc:407] Connection header received
0926 06:56:34.744706 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:34.744707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:34.744717 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:34.744778 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:34.745337 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:34.745960 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:34.747289 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:34.747567 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:34.747656 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:34.750043 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:34.750045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:34.750045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:34.750051 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:34.751517 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:34.752033 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:34.752034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:34.752034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:34.752081 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:34.752392 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:34.752393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:34.752393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:34.752498 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:34.752547 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:34.752896 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 06:56:34.752963 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:56:34.753283 19035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:34.743517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:40959 (local address 127.0.191.193:54185)
0926 06:56:34.743685 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:34.743727 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:34.744616 (+ 889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:34.744836 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:34.744840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:34.745082 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:34.745264 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:34.745269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:34.746060 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:34.746062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:34.747209 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:34.747212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:34.747663 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:34.747664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:34.747805 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:34.748364 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:56:34.748373 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:34.749942 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:34.751602 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:34.751610 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:34.751612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:34.751954 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:34.752151 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:34.752152 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:34.752152 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:34.752310 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:34.752567 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:34.752574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:34.752755 (+ 181us) client_negotiation.cc:769] Sending connection context
0926 06:56:34.752881 (+ 126us) client_negotiation.cc:240] Negotiation successful
0926 06:56:34.753057 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260926 06:56:34.753877 19033 heartbeater.cc:347] Connected to a master server at 127.0.191.254:40959
I20260926 06:56:34.754009 19033 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:34.754192 19033 heartbeater.cc:511] Master 127.0.191.254:40959 requested a full tablet report, sending...
I20260926 06:56:34.754729 18842 ts_manager.cc:194] Registered new tserver with Master: e3e732dc052346e9a008cafad975a5da (127.0.191.193:36419)
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
I20260926 06:56:34.755379 18842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:54185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:56:34.772817 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:40959
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:35859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:34.853888 19038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:34.854151 19038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:34.854189 19038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:34.855317 19038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:34.855381 19038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:34.855423 19038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:56:34.856875 19038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:35859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:40959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:34.857143 19038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:34.857391 19038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:34.862322 19043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:34.862422 19044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:34.862341 19046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:34.862886 19038 server_base.cc:1034] running on GCE node
I20260926 06:56:34.863076 19038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:34.863335 19038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:34.864512 19038 hybrid_clock.cc:648] HybridClock initialized: now 1790405794864498 us; error 32 us; skew 500 ppm
Sep 26 06:56:34 dist-test-slave-ws74 krb5kdc[18797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:34.867182 19038 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:56:34.867566 19038 webserver.cc:460] Webserver started at http://127.0.191.194:44371/ using document root <none> and password file <none>
I20260926 06:56:34.867743 19038 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:34.867782 19038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:34.867882 19038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:34.868901 19038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25"
format_stamp: "Formatted at 2026-09-26 06:56:34 on dist-test-slave-ws74"
server_key: "3e1c800c1004cc201dea1b3e5d603dd9"
server_key_iv: "c92e9a517cd9987bceb4ea75b7106bcb"
server_key_version: "encryptionkey@0"
I20260926 06:56:34.869199 19038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25"
format_stamp: "Formatted at 2026-09-26 06:56:34 on dist-test-slave-ws74"
server_key: "3e1c800c1004cc201dea1b3e5d603dd9"
server_key_iv: "c92e9a517cd9987bceb4ea75b7106bcb"
server_key_version: "encryptionkey@0"
I20260926 06:56:34.870543 19038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:56:34.871624 19053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:34.871902 19038 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:56:34.872020 19038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25"
format_stamp: "Formatted at 2026-09-26 06:56:34 on dist-test-slave-ws74"
server_key: "3e1c800c1004cc201dea1b3e5d603dd9"
server_key_iv: "c92e9a517cd9987bceb4ea75b7106bcb"
server_key_version: "encryptionkey@0"
I20260926 06:56:34.872098 19038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:34.896680 19038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:34.899057 19038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:34.899255 19038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:34.899523 19038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:34.899832 19038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:34.899897 19038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:34.899942 19038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:34.899966 19038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:34.906176 19038 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:44743
I20260926 06:56:34.906222 19166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:44743 every 8 connection(s)
I20260926 06:56:34.906543 19038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:56:34.907765 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 19038
I20260926 06:56:34.907883 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:56:34.908073 767 external_mini_cluster.cc:1428] Setting key 1436aa263a2ee60a37c03114774a17f3
Sep 26 06:56:34 dist-test-slave-ws74 krb5kdc[18797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:34.916610 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:34.907278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:37735 (local address 127.0.191.254:40959)
0926 06:56:34.907336 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:34.907338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:34.907532 (+ 194us) server_negotiation.cc:407] Connection header received
0926 06:56:34.908651 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:34.908652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:34.908662 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:34.908723 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:34.909359 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:34.909883 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:34.910894 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:34.911213 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:34.911279 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:34.913749 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:34.913750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:34.913751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:34.913756 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:34.915116 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:34.915620 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:34.915621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:34.915621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:34.915682 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:34.915976 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:34.915976 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:34.915976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:34.916074 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:34.916121 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:34.916477 (+ 356us) server_negotiation.cc:299] Negotiation successful
0926 06:56:34.916547 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 06:56:34.916872 19169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:34.907226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:40959 (local address 127.0.191.194:37735)
0926 06:56:34.907392 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:34.907433 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:34.908550 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:34.908806 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:34.908809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:34.909075 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:34.909266 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:34.909270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:34.909979 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:34.909980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:34.910800 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:34.910802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:34.911311 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:34.911312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:34.911444 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:34.911991 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:56:34.912000 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:34.913647 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:34.915202 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:34.915209 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:34.915211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:34.915537 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:34.915753 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:34.915754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:34.915755 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:34.915905 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:34.916139 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:34.916146 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:34.916316 (+ 170us) client_negotiation.cc:769] Sending connection context
0926 06:56:34.916462 (+ 146us) client_negotiation.cc:240] Negotiation successful
0926 06:56:34.916638 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
I20260926 06:56:34.917397 19167 heartbeater.cc:347] Connected to a master server at 127.0.191.254:40959
I20260926 06:56:34.917491 19167 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:34.917683 19167 heartbeater.cc:511] Master 127.0.191.254:40959 requested a full tablet report, sending...
I20260926 06:56:34.918097 18842 ts_manager.cc:194] Registered new tserver with Master: 7a347bbf1ac24e9ba5d4c0cfaafedc25 (127.0.191.194:44743)
I20260926 06:56:34.918476 18842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:37735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:56:34.933754 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:40959
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:35859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:35.015264 19172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:35.015520 19172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:35.015561 19172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:35.016738 19172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:35.016812 19172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:35.016856 19172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:56:35.018468 19172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:35859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:40959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:35.018747 19172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:35.018980 19172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:35.024087 19180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:35.024088 19177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:35.024318 19172 server_base.cc:1034] running on GCE node
W20260926 06:56:35.024223 19178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:35.024641 19172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:35.024873 19172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:35.026031 19172 hybrid_clock.cc:648] HybridClock initialized: now 1790405795026030 us; error 37 us; skew 500 ppm
Sep 26 06:56:35 dist-test-slave-ws74 krb5kdc[18797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405795, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:35.028584 19172 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:56:35.029004 19172 webserver.cc:460] Webserver started at http://127.0.191.195:46165/ using document root <none> and password file <none>
I20260926 06:56:35.029227 19172 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:35.029292 19172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:35.029454 19172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:35.030503 19172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "074e4af9237d4f18961d4c343c6f65ab"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "0c2889ce614812bf2ebdc91eecb21339"
server_key_iv: "d1810993639fe6cd9c028a94a2746c88"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.030866 19172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "074e4af9237d4f18961d4c343c6f65ab"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "0c2889ce614812bf2ebdc91eecb21339"
server_key_iv: "d1810993639fe6cd9c028a94a2746c88"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.032172 19172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:56:35.032961 19187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:35.033123 19172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:56:35.033206 19172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "074e4af9237d4f18961d4c343c6f65ab"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "0c2889ce614812bf2ebdc91eecb21339"
server_key_iv: "d1810993639fe6cd9c028a94a2746c88"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.033260 19172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:35.046110 19172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:35.048604 19172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:35.048849 19172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:35.049101 19172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:35.049480 19172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:35.049527 19172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:35.049552 19172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:35.049566 19172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:35.055689 19172 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:37027
I20260926 06:56:35.055758 19300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:37027 every 8 connection(s)
I20260926 06:56:35.056115 19172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:56:35.058094 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 19172
I20260926 06:56:35.058210 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:56:35.058451 767 external_mini_cluster.cc:1428] Setting key 2602a3e44b6238950497e334c6983913
Sep 26 06:56:35 dist-test-slave-ws74 krb5kdc[18797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405795, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:35.065909 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.056946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:52657 (local address 127.0.191.254:40959)
0926 06:56:35.057022 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.057025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.057271 (+ 246us) server_negotiation.cc:407] Connection header received
0926 06:56:35.058436 (+ 1165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.058437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.058451 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.058499 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:35.059238 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.059763 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.060775 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.060973 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.061060 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.063290 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:35.063291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:35.063291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:35.063298 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:35.064729 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:35.065086 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:35.065086 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:35.065087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:35.065136 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:35.065335 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:35.065335 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:35.065335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:35.065492 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:35.065544 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.065773 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.065841 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 06:56:35.066059 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.056918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:40959 (local address 127.0.191.195:52657)
0926 06:56:35.057131 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:35.057169 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:35.058306 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:35.058570 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:35.058574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:35.058930 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:35.059143 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.059148 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.059869 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.059870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.060686 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.060688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.061068 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.061069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.061199 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.061757 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:56:35.061765 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:35.063179 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:35.064800 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:35.064806 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:35.064807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:35.065019 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:35.065191 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:35.065191 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:35.065191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:35.065283 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:35.065547 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:35.065551 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:35.065662 (+ 111us) client_negotiation.cc:769] Sending connection context
0926 06:56:35.065748 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 06:56:35.065866 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":117,"threads_started":1}
I20260926 06:56:35.066586 19301 heartbeater.cc:347] Connected to a master server at 127.0.191.254:40959
I20260926 06:56:35.066709 19301 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:35.066896 19301 heartbeater.cc:511] Master 127.0.191.254:40959 requested a full tablet report, sending...
I20260926 06:56:35.067368 18842 ts_manager.cc:194] Registered new tserver with Master: 074e4af9237d4f18961d4c343c6f65ab (127.0.191.195:37027)
I20260926 06:56:35.067929 18842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:52657
I20260926 06:56:35.070006 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:56:35.078692 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.071690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.0.191.254:40959)
0926 06:56:35.071744 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.071746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.071891 (+ 145us) server_negotiation.cc:407] Connection header received
0926 06:56:35.071968 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.071968 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.071997 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.072080 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:35.072722 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.073280 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.074202 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.074405 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.074484 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.075920 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:35.075921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:35.075922 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:35.075929 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:35.077498 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:35.077879 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:35.077880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:35.077880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:35.077933 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:35.078156 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:35.078157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:35.078157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:35.078289 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:35.078404 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.078534 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.078595 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:56:35.080066 18842 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:56:35.081573 18842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:56:35.088028 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.083621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:44743 (local address 127.0.0.1:49606)
0926 06:56:35.083771 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:35.083807 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:35.083874 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:35.084564 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:35.084565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:35.084566 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:35.084621 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.084623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.085899 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.085900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.086899 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.086901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.087687 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.087688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.087788 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.087808 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 06:56:35.087888 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 06:56:35.087941 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":43,"thread_start_us":46,"threads_started":1}
I20260926 06:56:35.088032 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.083944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.0.191.194:44743)
0926 06:56:35.084368 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.084372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.084389 (+ 17us) server_negotiation.cc:407] Connection header received
0926 06:56:35.084419 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.084420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.084431 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.084566 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:35.084741 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.085811 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.086980 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.087602 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.087676 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.087729 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.087912 (+ 183us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.087951 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":71,"threads_started":1}
I20260926 06:56:35.088708 19313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.083634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:36419 (local address 127.0.0.1:46178)
0926 06:56:35.083975 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:35.083983 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:35.084031 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:35.084315 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:35.084319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:35.084322 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:35.084420 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.084421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.085839 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.085840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.087211 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.087213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.088472 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.088474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.088540 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.088545 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:56:35.088603 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 06:56:35.088644 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":47,"thread_start_us":86,"threads_started":1}
I20260926 06:56:35.088734 19316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.083882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46178 (local address 127.0.191.193:36419)
0926 06:56:35.084061 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.084066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.084076 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:56:35.084120 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.084122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.084137 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.084223 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:35.084561 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.085734 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.087297 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.088019 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.088089 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.088147 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.088620 (+ 473us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.088661 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260926 06:56:35.088846 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.083623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:37027 (local address 127.0.0.1:34310)
0926 06:56:35.083854 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:35.083863 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:35.083904 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:35.084322 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:35.084323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:35.084325 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:35.084418 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.084420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.086649 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.086650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.087704 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.087706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.088613 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.088615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.088679 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.088684 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:56:35.088717 (+ 33us) client_negotiation.cc:240] Negotiation successful
0926 06:56:35.088756 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260926 06:56:35.088901 19101 tablet_service.cc:1467] Processing CreateTablet for tablet a6d6c4199e8a41708ee4e91ac1b4ccd8 (DEFAULT_TABLE table=test-table [id=cc5b16c54fbd4e95844ecda876c638cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:35.089296 19315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.083766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34310 (local address 127.0.191.195:37027)
0926 06:56:35.084140 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.084144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.084152 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:56:35.084187 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.084189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.084204 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.084279 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:35.085453 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.086520 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.087821 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.088503 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.088937 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.089001 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.089168 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.089214 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":81,"threads_started":1}
I20260926 06:56:35.089298 19101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d6c4199e8a41708ee4e91ac1b4ccd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:35.089921 19235 tablet_service.cc:1467] Processing CreateTablet for tablet a6d6c4199e8a41708ee4e91ac1b4ccd8 (DEFAULT_TABLE table=test-table [id=cc5b16c54fbd4e95844ecda876c638cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:35.090214 19235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d6c4199e8a41708ee4e91ac1b4ccd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:35.092315 19318 tablet_bootstrap.cc:492] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25: Bootstrap starting.
I20260926 06:56:35.092721 18967 tablet_service.cc:1467] Processing CreateTablet for tablet a6d6c4199e8a41708ee4e91ac1b4ccd8 (DEFAULT_TABLE table=test-table [id=cc5b16c54fbd4e95844ecda876c638cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:35.092990 19318 tablet_bootstrap.cc:654] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:35.092988 19319 tablet_bootstrap.cc:492] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab: Bootstrap starting.
I20260926 06:56:35.093072 18967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d6c4199e8a41708ee4e91ac1b4ccd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:35.093562 19318 log.cc:824] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:35.093720 19319 tablet_bootstrap.cc:654] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:35.094301 19318 tablet_bootstrap.cc:492] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25: No bootstrap required, opened a new log
I20260926 06:56:35.094334 19319 log.cc:824] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:35.094383 19318 ts_tablet_manager.cc:1397] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:56:35.095141 19319 tablet_bootstrap.cc:492] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab: No bootstrap required, opened a new log
I20260926 06:56:35.095223 19319 ts_tablet_manager.cc:1397] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:56:35.096149 19318 raft_consensus.cc:348] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 44743 } } peers { permanent_uuid: "074e4af9237d4f18961d4c343c6f65ab" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37027 } } peers { permanent_uuid: "e3e732dc052346e9a008cafad975a5da" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 36419 } }
I20260926 06:56:35.096310 19318 raft_consensus.cc:374] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:35.096354 19318 raft_consensus.cc:729] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a347bbf1ac24e9ba5d4c0cfaafedc25, State: Initialized, Role: FOLLOWER
I20260926 06:56:35.096464 19318 consensus_queue.cc:260] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 44743 } } peers { permanent_uuid: "074e4af9237d4f18961d4c343c6f65ab" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37027 } } peers { permanent_uuid: "e3e732dc052346e9a008cafad975a5da" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 36419 } }
I20260926 06:56:35.096756 19318 ts_tablet_manager.cc:1428] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:56:35.096753 19322 tablet_bootstrap.cc:492] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da: Bootstrap starting.
I20260926 06:56:35.096916 19167 heartbeater.cc:503] Master 127.0.191.254:40959 was elected leader, sending a full tablet report...
I20260926 06:56:35.097509 19322 tablet_bootstrap.cc:654] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:35.097527 19319 raft_consensus.cc:348] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 44743 } } peers { permanent_uuid: "074e4af9237d4f18961d4c343c6f65ab" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37027 } } peers { permanent_uuid: "e3e732dc052346e9a008cafad975a5da" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 36419 } }
I20260926 06:56:35.097704 19319 raft_consensus.cc:374] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:35.097744 19319 raft_consensus.cc:729] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 074e4af9237d4f18961d4c343c6f65ab, State: Initialized, Role: FOLLOWER
I20260926 06:56:35.097916 19319 consensus_queue.cc:260] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 44743 } } peers { permanent_uuid: "074e4af9237d4f18961d4c343c6f65ab" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37027 } } peers { permanent_uuid: "e3e732dc052346e9a008cafad975a5da" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 36419 } }
I20260926 06:56:35.098119 19322 log.cc:824] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:35.098245 19319 ts_tablet_manager.cc:1428] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260926 06:56:35.098276 19301 heartbeater.cc:503] Master 127.0.191.254:40959 was elected leader, sending a full tablet report...
I20260926 06:56:35.098963 19322 tablet_bootstrap.cc:492] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da: No bootstrap required, opened a new log
I20260926 06:56:35.099040 19322 ts_tablet_manager.cc:1397] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:56:35.100572 19322 raft_consensus.cc:348] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 44743 } } peers { permanent_uuid: "074e4af9237d4f18961d4c343c6f65ab" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37027 } } peers { permanent_uuid: "e3e732dc052346e9a008cafad975a5da" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 36419 } }
I20260926 06:56:35.100711 19322 raft_consensus.cc:374] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:35.100734 19322 raft_consensus.cc:729] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e732dc052346e9a008cafad975a5da, State: Initialized, Role: FOLLOWER
I20260926 06:56:35.100878 19322 consensus_queue.cc:260] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 44743 } } peers { permanent_uuid: "074e4af9237d4f18961d4c343c6f65ab" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37027 } } peers { permanent_uuid: "e3e732dc052346e9a008cafad975a5da" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 36419 } }
I20260926 06:56:35.101157 19322 ts_tablet_manager.cc:1428] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:56:35.101168 19033 heartbeater.cc:503] Master 127.0.191.254:40959 was elected leader, sending a full tablet report...
W20260926 06:56:35.157181 19168 tablet.cc:2369] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:56:35.243535 19034 tablet.cc:2369] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:56:35.306687 19302 tablet.cc:2369] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:56:35.335786 19324 raft_consensus.cc:482] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:56:35.335955 19324 raft_consensus.cc:504] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 44743 } } peers { permanent_uuid: "074e4af9237d4f18961d4c343c6f65ab" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37027 } } peers { permanent_uuid: "e3e732dc052346e9a008cafad975a5da" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 36419 } }
I20260926 06:56:35.336324 19324 leader_election.cc:288] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a347bbf1ac24e9ba5d4c0cfaafedc25 (127.0.191.194:44743), e3e732dc052346e9a008cafad975a5da (127.0.191.193:36419)
I20260926 06:56:35.339280 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.336684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:57881 (local address 127.0.191.194:44743)
0926 06:56:35.336732 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.336734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.336746 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:56:35.336778 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.336779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.336789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.336850 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:35.337028 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.337596 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.338565 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.338926 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.339016 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.339024 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.339174 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.339213 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:56:35.339283 19316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.336620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:40597 (local address 127.0.191.193:36419)
0926 06:56:35.336691 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.336694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.336805 (+ 111us) server_negotiation.cc:407] Connection header received
0926 06:56:35.336867 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.336868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.336877 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.336936 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:35.337125 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.337609 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.338567 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.338931 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.339016 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.339024 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.339187 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.339221 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:56:35.339293 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.336541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:44743 (local address 127.0.191.195:57881)
0926 06:56:35.336601 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:35.336612 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:35.336688 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:35.336865 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:35.336865 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:35.336867 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:35.336946 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.336948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.337682 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.337683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.338469 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.338471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.339013 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.339014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.339090 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.339094 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:56:35.339167 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 06:56:35.339208 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 06:56:35.339293 19327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.336559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:36419 (local address 127.0.191.195:40597)
0926 06:56:35.336732 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:35.336742 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:35.336807 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:35.336951 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:35.336952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:35.336953 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:35.337057 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.337058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.337714 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.337714 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.338471 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.338473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.339024 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.339025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.339096 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.339099 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:56:35.339178 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 06:56:35.339215 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":106,"threads_started":1}
I20260926 06:56:35.339617 19121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d6c4199e8a41708ee4e91ac1b4ccd8" candidate_uuid: "074e4af9237d4f18961d4c343c6f65ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25" is_pre_election: true
I20260926 06:56:35.339654 18987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d6c4199e8a41708ee4e91ac1b4ccd8" candidate_uuid: "074e4af9237d4f18961d4c343c6f65ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e732dc052346e9a008cafad975a5da" is_pre_election: true
I20260926 06:56:35.339771 19121 raft_consensus.cc:2454] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 074e4af9237d4f18961d4c343c6f65ab in term 0.
I20260926 06:56:35.339828 18987 raft_consensus.cc:2454] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 074e4af9237d4f18961d4c343c6f65ab in term 0.
I20260926 06:56:35.339980 19191 leader_election.cc:302] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 074e4af9237d4f18961d4c343c6f65ab, 7a347bbf1ac24e9ba5d4c0cfaafedc25; no voters:
I20260926 06:56:35.340130 19324 raft_consensus.cc:2781] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:56:35.340188 19324 raft_consensus.cc:482] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:56:35.340220 19324 raft_consensus.cc:3037] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:35.340888 19324 raft_consensus.cc:504] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 44743 } } peers { permanent_uuid: "074e4af9237d4f18961d4c343c6f65ab" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37027 } } peers { permanent_uuid: "e3e732dc052346e9a008cafad975a5da" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 36419 } }
I20260926 06:56:35.341033 19324 leader_election.cc:288] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [CANDIDATE]: Term 1 election: Requested vote from peers 7a347bbf1ac24e9ba5d4c0cfaafedc25 (127.0.191.194:44743), e3e732dc052346e9a008cafad975a5da (127.0.191.193:36419)
I20260926 06:56:35.341182 19121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d6c4199e8a41708ee4e91ac1b4ccd8" candidate_uuid: "074e4af9237d4f18961d4c343c6f65ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25"
I20260926 06:56:35.341253 19121 raft_consensus.cc:3037] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:35.341220 18987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d6c4199e8a41708ee4e91ac1b4ccd8" candidate_uuid: "074e4af9237d4f18961d4c343c6f65ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e732dc052346e9a008cafad975a5da"
I20260926 06:56:35.341325 18987 raft_consensus.cc:3037] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:35.341954 19121 raft_consensus.cc:2454] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 074e4af9237d4f18961d4c343c6f65ab in term 1.
I20260926 06:56:35.342139 18987 raft_consensus.cc:2454] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 074e4af9237d4f18961d4c343c6f65ab in term 1.
I20260926 06:56:35.342133 19191 leader_election.cc:302] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 074e4af9237d4f18961d4c343c6f65ab, 7a347bbf1ac24e9ba5d4c0cfaafedc25; no voters:
I20260926 06:56:35.342355 19324 raft_consensus.cc:2781] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:35.342483 19324 raft_consensus.cc:686] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [term 1 LEADER]: Becoming Leader. State: Replica: 074e4af9237d4f18961d4c343c6f65ab, State: Running, Role: LEADER
I20260926 06:56:35.342581 19324 consensus_queue.cc:237] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 44743 } } peers { permanent_uuid: "074e4af9237d4f18961d4c343c6f65ab" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37027 } } peers { permanent_uuid: "e3e732dc052346e9a008cafad975a5da" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 36419 } }
I20260926 06:56:35.343427 18842 catalog_manager.cc:5495] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 074e4af9237d4f18961d4c343c6f65ab (127.0.191.195). New cstate: current_term: 1 leader_uuid: "074e4af9237d4f18961d4c343c6f65ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 44743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "074e4af9237d4f18961d4c343c6f65ab" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3e732dc052346e9a008cafad975a5da" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 36419 } health_report { overall_health: UNKNOWN } } }
I20260926 06:56:35.369787 19315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.367434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34318 (local address 127.0.191.195:37027)
0926 06:56:35.367502 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.367504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.367519 (+ 15us) server_negotiation.cc:407] Connection header received
0926 06:56:35.367550 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.367550 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.367567 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.367650 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:56:35.367757 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.368231 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.368997 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.369191 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.369267 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.369451 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:56:35.369536 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:56:35.369646 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.369684 (+ 38us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.369695 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:56:35.371608 19121 raft_consensus.cc:1261] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25 [term 1 FOLLOWER]: Refusing update from remote peer 074e4af9237d4f18961d4c343c6f65ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:56:35.371613 18987 raft_consensus.cc:1261] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P e3e732dc052346e9a008cafad975a5da [term 1 FOLLOWER]: Refusing update from remote peer 074e4af9237d4f18961d4c343c6f65ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:56:35.371985 19328 consensus_queue.cc:1035] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a347bbf1ac24e9ba5d4c0cfaafedc25" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 44743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:56:35.372160 19324 consensus_queue.cc:1035] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3e732dc052346e9a008cafad975a5da" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 36419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:56:35.374490 19332 mvcc.cc:205] Tried to move back new op lower bound from 7333502137839169536 to 7333502137723944960. Current Snapshot: MvccSnapshot[applied={T|T < 7333502137839169536}]
I20260926 06:56:35.374634 19331 mvcc.cc:205] Tried to move back new op lower bound from 7333502137839169536 to 7333502137723944960. Current Snapshot: MvccSnapshot[applied={T|T < 7333502137839169536}]
I20260926 06:56:35.375243 19333 mvcc.cc:205] Tried to move back new op lower bound from 7333502137839169536 to 7333502137723944960. Current Snapshot: MvccSnapshot[applied={T|T < 7333502137839169536}]
I20260926 06:56:35.376890 18842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:56:35.376984 18842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:56:35.377753 18842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40691525a2bc41d9b4b3ffe02c9aeded: Sending DeleteTablet for 3 replicas of tablet a6d6c4199e8a41708ee4e91ac1b4ccd8
I20260926 06:56:35.378078 18967 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d6c4199e8a41708ee4e91ac1b4ccd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:35 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:46178
I20260926 06:56:35.378101 19101 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d6c4199e8a41708ee4e91ac1b4ccd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:35 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:49606
I20260926 06:56:35.378094 19235 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d6c4199e8a41708ee4e91ac1b4ccd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:35 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:34310
I20260926 06:56:35.378355 19342 tablet_replica.cc:330] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab: stopping tablet replica
I20260926 06:56:35.378453 19342 raft_consensus.cc:2229] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:56:35.378517 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 18904
I20260926 06:56:35.378544 19342 raft_consensus.cc:2258] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:56:35.379340 19342 ts_tablet_manager.cc:1905] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:56:35.380903 19340 tablet_replica.cc:330] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25: stopping tablet replica
I20260926 06:56:35.381014 19340 raft_consensus.cc:2229] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:56:35.381122 19340 raft_consensus.cc:2258] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:56:35.381196 19342 ts_tablet_manager.cc:1918] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:56:35.381300 19342 log.cc:1196] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/a6d6c4199e8a41708ee4e91ac1b4ccd8
I20260926 06:56:35.381572 19340 ts_tablet_manager.cc:1905] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:56:35.381743 19342 ts_tablet_manager.cc:1939] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 074e4af9237d4f18961d4c343c6f65ab: Deleting consensus metadata
I20260926 06:56:35.382143 18826 catalog_manager.cc:4841] TS 074e4af9237d4f18961d4c343c6f65ab (127.0.191.195:37027): tablet a6d6c4199e8a41708ee4e91ac1b4ccd8 (table test-table [id=cc5b16c54fbd4e95844ecda876c638cf]) successfully deleted
I20260926 06:56:35.383112 19340 ts_tablet_manager.cc:1918] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:56:35.383185 19340 log.cc:1196] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/a6d6c4199e8a41708ee4e91ac1b4ccd8
I20260926 06:56:35.383443 19340 ts_tablet_manager.cc:1939] T a6d6c4199e8a41708ee4e91ac1b4ccd8 P 7a347bbf1ac24e9ba5d4c0cfaafedc25: Deleting consensus metadata
I20260926 06:56:35.383843 18829 catalog_manager.cc:4841] TS 7a347bbf1ac24e9ba5d4c0cfaafedc25 (127.0.191.194:44743): tablet a6d6c4199e8a41708ee4e91ac1b4ccd8 (table test-table [id=cc5b16c54fbd4e95844ecda876c638cf]) successfully deleted
W20260926 06:56:35.384431 18828 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:36419 (error 108)
I20260926 06:56:35.384971 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 19038
I20260926 06:56:35.384969 19313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.384831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:36419 (local address 127.0.0.1:46194)
0926 06:56:35.384891 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:35.384936 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:36419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 06:56:35.386579 18828 catalog_manager.cc:4584] TS e3e732dc052346e9a008cafad975a5da (127.0.191.193:36419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6d6c4199e8a41708ee4e91ac1b4ccd8: Network error: Client connection negotiation failed: client connection to 127.0.191.193:36419: connect: Connection refused (error 111)
I20260926 06:56:35.389753 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 19172
I20260926 06:56:35.394413 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 18810
2026-09-26T06:56:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2155 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T06:56:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:35Z Disabled control of system clock
I20260926 06:56:35.421245 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33579
--webserver_interface=127.0.191.254
--webserver_port=0
--builtin_ntp_servers=127.0.191.212:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:33579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:56:35.500499 19347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:35.500717 19347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:35.500737 19347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:35.501931 19347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:35.501971 19347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:35.501984 19347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:35.501999 19347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:35.502036 19347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:35.503407 19347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:33579
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:33579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:35.503660 19347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:35.503943 19347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:35.508798 19353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:35.508831 19355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:35.508824 19347 server_base.cc:1034] running on GCE node
W20260926 06:56:35.508857 19352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:35.509162 19347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:35.509505 19347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:35.510668 19347 hybrid_clock.cc:648] HybridClock initialized: now 1790405795510668 us; error 36 us; skew 500 ppm
I20260926 06:56:35.511953 19347 webserver.cc:460] Webserver started at http://127.0.191.254:41997/ using document root <none> and password file <none>
I20260926 06:56:35.512207 19347 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:35.512272 19347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:35.512422 19347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:35.513574 19347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "41e20a484d1f47609688b068ee6385f3"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "e260944b1ed7084d5836da9c914bd14a"
server_key_iv: "e18d4ca59f79c7e1c49fc50a798276f1"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.513933 19347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "41e20a484d1f47609688b068ee6385f3"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "e260944b1ed7084d5836da9c914bd14a"
server_key_iv: "e18d4ca59f79c7e1c49fc50a798276f1"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.515261 19347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:56:35.516054 19361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:35.516259 19347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:35.516356 19347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "41e20a484d1f47609688b068ee6385f3"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "e260944b1ed7084d5836da9c914bd14a"
server_key_iv: "e18d4ca59f79c7e1c49fc50a798276f1"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.516435 19347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:35.529747 19347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:35.530057 19347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:35.530205 19347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:35.534303 19347 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:33579
I20260926 06:56:35.534356 19413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:33579 every 8 connection(s)
I20260926 06:56:35.534719 19347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:35.535382 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 19347
I20260926 06:56:35.535486 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:35.535732 767 external_mini_cluster.cc:1428] Setting key c84abe6134fd2267721cf0b6bb61fb60
I20260926 06:56:35.535866 19414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:35.539628 19414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3: Bootstrap starting.
I20260926 06:56:35.540373 19414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:35.540700 19414 log.cc:824] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:35.541555 19414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3: No bootstrap required, opened a new log
I20260926 06:56:35.541596 19417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.536308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46102 (local address 127.0.191.254:33579)
0926 06:56:35.536522 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.536529 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.536553 (+ 24us) server_negotiation.cc:407] Connection header received
0926 06:56:35.536981 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.536987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.537051 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.537175 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:35.537489 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.538323 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.539543 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.539987 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.540079 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.540873 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:35.540877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:35.540878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:35.541037 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:35.541068 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:35.541078 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:35.541126 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.541256 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.541334 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":105,"threads_started":1}
I20260926 06:56:35.543428 19414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e20a484d1f47609688b068ee6385f3" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33579 } }
I20260926 06:56:35.543560 19414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:35.543583 19414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e20a484d1f47609688b068ee6385f3, State: Initialized, Role: FOLLOWER
I20260926 06:56:35.543694 19414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e20a484d1f47609688b068ee6385f3" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33579 } }
I20260926 06:56:35.543761 19414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:35.543781 19414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:35.543809 19414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:35.544294 19414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e20a484d1f47609688b068ee6385f3" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33579 } }
I20260926 06:56:35.544374 19414 leader_election.cc:302] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41e20a484d1f47609688b068ee6385f3; no voters:
I20260926 06:56:35.544548 19414 leader_election.cc:288] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:35.544741 19419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:35.544823 19414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:35.544929 19419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [term 1 LEADER]: Becoming Leader. State: Replica: 41e20a484d1f47609688b068ee6385f3, State: Running, Role: LEADER
I20260926 06:56:35.545032 19419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e20a484d1f47609688b068ee6385f3" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33579 } }
I20260926 06:56:35.545470 19421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41e20a484d1f47609688b068ee6385f3. Latest consensus state: current_term: 1 leader_uuid: "41e20a484d1f47609688b068ee6385f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e20a484d1f47609688b068ee6385f3" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33579 } } }
I20260926 06:56:35.545575 19421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:35.545485 19420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41e20a484d1f47609688b068ee6385f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e20a484d1f47609688b068ee6385f3" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 33579 } } }
I20260926 06:56:35.545907 19420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:35.546121 19428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:35.546636 19428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:35.548013 19428 catalog_manager.cc:1347] Generated new cluster ID: 0d42065490cb4ca5a4961fe11e3f05c3
I20260926 06:56:35.548077 19428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:35.555022 19428 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:35.555617 19428 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:35.562420 19428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3: Generated new TSK 0
I20260926 06:56:35.562615 19428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:35.565903 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33579
--builtin_ntp_servers=127.0.191.212:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:56:35.645900 19438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:35.646101 19438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:35.646152 19438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:35.647394 19438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:35.647437 19438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:35.647464 19438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:56:35.648823 19438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:35.649058 19438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:35.649407 19438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:56:35.654397 19438 server_base.cc:1034] running on GCE node
W20260926 06:56:35.654366 19446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:35.654462 19444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:35.654491 19443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:35.654860 19438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:35.655136 19438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:35.656356 19438 hybrid_clock.cc:648] HybridClock initialized: now 1790405795656308 us; error 30 us; skew 500 ppm
I20260926 06:56:35.657923 19438 webserver.cc:460] Webserver started at http://127.0.191.193:44853/ using document root <none> and password file <none>
I20260926 06:56:35.658219 19438 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:35.658270 19438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:35.658466 19438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:35.659750 19438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "4d43deb998a14c5e97a90ac68abff7ac"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "bb5a31d35d53e2bf40fe8e01f75d1a29"
server_key_iv: "8c57c0c6a0d581ddc8b06220d99918c1"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.660164 19438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "4d43deb998a14c5e97a90ac68abff7ac"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "bb5a31d35d53e2bf40fe8e01f75d1a29"
server_key_iv: "8c57c0c6a0d581ddc8b06220d99918c1"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.661712 19438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260926 06:56:35.662582 19452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:35.662827 19438 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:35.662920 19438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "4d43deb998a14c5e97a90ac68abff7ac"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "bb5a31d35d53e2bf40fe8e01f75d1a29"
server_key_iv: "8c57c0c6a0d581ddc8b06220d99918c1"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.662987 19438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:35.683457 19438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:35.683740 19438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:35.683846 19438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:35.684044 19438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:35.684393 19438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:35.684427 19438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:35.684497 19438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:35.684523 19438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:35.690346 19438 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:39753
I20260926 06:56:35.690500 19565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:39753 every 8 connection(s)
I20260926 06:56:35.690845 19438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:56:35.695986 19417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.691660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:56107 (local address 127.0.191.254:33579)
0926 06:56:35.691739 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.691741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.692004 (+ 263us) server_negotiation.cc:407] Connection header received
0926 06:56:35.692468 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.692469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.692481 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.692545 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:35.693189 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.693770 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.694715 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.694889 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.694936 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.695781 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:35.695782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:35.695783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:35.695786 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:35.695797 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:35.695800 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:35.695836 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.695902 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.695929 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 06:56:35.696151 19568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.691651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33579 (local address 127.0.191.193:56107)
0926 06:56:35.691858 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:35.691900 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:35.692386 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:35.692597 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:35.692600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:35.692607 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:35.693116 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.693121 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.693848 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.693849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.694641 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.694643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.694965 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.694966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.695077 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.695654 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:56:35.695655 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:35.695684 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:56:35.695706 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:35.695837 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:35.695840 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:35.695842 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:56:35.695890 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 06:56:35.695964 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260926 06:56:35.696655 19566 heartbeater.cc:347] Connected to a master server at 127.0.191.254:33579
I20260926 06:56:35.696735 19566 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:35.696962 19566 heartbeater.cc:511] Master 127.0.191.254:33579 requested a full tablet report, sending...
I20260926 06:56:35.697549 19378 ts_manager.cc:194] Registered new tserver with Master: 4d43deb998a14c5e97a90ac68abff7ac (127.0.191.193:39753)
I20260926 06:56:35.698165 19378 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.191.193:56107
I20260926 06:56:35.700905 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 19438
I20260926 06:56:35.701041 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:56:35.701261 767 external_mini_cluster.cc:1428] Setting key 91701bf97779c8956ad4a42bdd773003
I20260926 06:56:35.702240 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33579
--builtin_ntp_servers=127.0.191.212:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:56:35.781426 19569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:35.781625 19569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:35.781646 19569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:35.782752 19569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:35.782785 19569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:35.782809 19569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:56:35.784070 19569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:35.784240 19569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:35.784458 19569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:35.789273 19575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:35.789266 19574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:35.789268 19577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:35.789587 19569 server_base.cc:1034] running on GCE node
I20260926 06:56:35.789810 19569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:35.790053 19569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:35.791210 19569 hybrid_clock.cc:648] HybridClock initialized: now 1790405795791208 us; error 37 us; skew 500 ppm
I20260926 06:56:35.792371 19569 webserver.cc:460] Webserver started at http://127.0.191.194:37649/ using document root <none> and password file <none>
I20260926 06:56:35.792615 19569 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:35.792685 19569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:35.792819 19569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:35.793926 19569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "590b099edfc7430f92838e16ee843824"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "09e75d883308bae92743a7e775b587c3"
server_key_iv: "f6acead79290419fa68b0009189d1917"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.794327 19569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "590b099edfc7430f92838e16ee843824"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "09e75d883308bae92743a7e775b587c3"
server_key_iv: "f6acead79290419fa68b0009189d1917"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.795586 19569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:56:35.796376 19583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:35.796551 19569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:35.796643 19569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "590b099edfc7430f92838e16ee843824"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "09e75d883308bae92743a7e775b587c3"
server_key_iv: "f6acead79290419fa68b0009189d1917"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.796718 19569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:35.808853 19569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:35.809144 19569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:35.809291 19569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:35.809537 19569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:35.809844 19569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:35.809900 19569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:35.809942 19569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:35.809965 19569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:35.815709 19569 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:42811
I20260926 06:56:35.815749 19696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:42811 every 8 connection(s)
I20260926 06:56:35.816069 19569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:56:35.816821 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 19569
I20260926 06:56:35.816933 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:56:35.817118 767 external_mini_cluster.cc:1428] Setting key 23cd77a2192290c30d698dcd5f9fade9
I20260926 06:56:35.818267 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33579
--builtin_ntp_servers=127.0.191.212:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 06:56:35.822566 19417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.816831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:40769 (local address 127.0.191.254:33579)
0926 06:56:35.816885 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.816888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.817189 (+ 301us) server_negotiation.cc:407] Connection header received
0926 06:56:35.817871 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.817872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.817880 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.817931 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:35.818643 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.819465 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.820928 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.821251 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.821365 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.822229 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:35.822231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:35.822231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:35.822256 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:35.822273 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:35.822278 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:35.822348 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.822424 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.822450 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 06:56:35.822669 19699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.816753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33579 (local address 127.0.191.194:40769)
0926 06:56:35.817073 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:35.817112 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:35.817752 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:35.817994 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:35.817997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:35.818005 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:35.818558 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.818563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.819579 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.819580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.820852 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.820855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.821422 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.821423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.821551 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.822094 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:56:35.822094 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:35.822121 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:56:35.822142 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:35.822352 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:35.822356 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:35.822359 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:56:35.822406 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 06:56:35.822476 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":90,"threads_started":1}
I20260926 06:56:35.823202 19697 heartbeater.cc:347] Connected to a master server at 127.0.191.254:33579
I20260926 06:56:35.823316 19697 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:35.823487 19697 heartbeater.cc:511] Master 127.0.191.254:33579 requested a full tablet report, sending...
I20260926 06:56:35.823871 19378 ts_manager.cc:194] Registered new tserver with Master: 590b099edfc7430f92838e16ee843824 (127.0.191.194:42811)
I20260926 06:56:35.824415 19378 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.191.194:40769
W20260926 06:56:35.898834 19700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:35.899102 19700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:35.899128 19700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:35.900285 19700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:35.900324 19700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:35.900349 19700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:56:35.901630 19700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:35.901811 19700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:35.902050 19700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:35.907047 19705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:35.907092 19700 server_base.cc:1034] running on GCE node
W20260926 06:56:35.907034 19706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:35.907092 19708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:35.907470 19700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:35.907699 19700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:35.908842 19700 hybrid_clock.cc:648] HybridClock initialized: now 1790405795908827 us; error 31 us; skew 500 ppm
I20260926 06:56:35.910135 19700 webserver.cc:460] Webserver started at http://127.0.191.195:40267/ using document root <none> and password file <none>
I20260926 06:56:35.910377 19700 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:35.910421 19700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:35.910560 19700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:35.911780 19700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "d54012d20d55400c8c9e4828efa46ebe"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "9338b0f04cb3182d5afa3deccd7fbc39"
server_key_iv: "3063c91da6748d1bcb9398ab53b35822"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.912168 19700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "d54012d20d55400c8c9e4828efa46ebe"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "9338b0f04cb3182d5afa3deccd7fbc39"
server_key_iv: "3063c91da6748d1bcb9398ab53b35822"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.913640 19700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:56:35.914359 19714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:35.914553 19700 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:35.914618 19700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "d54012d20d55400c8c9e4828efa46ebe"
format_stamp: "Formatted at 2026-09-26 06:56:35 on dist-test-slave-ws74"
server_key: "9338b0f04cb3182d5afa3deccd7fbc39"
server_key_iv: "3063c91da6748d1bcb9398ab53b35822"
server_key_version: "encryptionkey@0"
I20260926 06:56:35.914708 19700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:35.932732 19700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:35.933068 19700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:35.933240 19700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:35.933537 19700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:35.933871 19700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:35.933903 19700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:35.933928 19700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:35.933975 19700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:35.939714 19700 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:45827
I20260926 06:56:35.939759 19827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:45827 every 8 connection(s)
I20260926 06:56:35.940096 19700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:56:35.943518 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 19700
I20260926 06:56:35.943607 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:56:35.943813 767 external_mini_cluster.cc:1428] Setting key b9129ada6699320770d017c6e7559613
I20260926 06:56:35.945681 19417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.940818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:34855 (local address 127.0.191.254:33579)
0926 06:56:35.940894 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.940896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.941165 (+ 269us) server_negotiation.cc:407] Connection header received
0926 06:56:35.941726 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.941727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.941738 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.941793 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:35.942462 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.942959 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.944067 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.944392 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.944454 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.945485 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:35.945486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:35.945486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:35.945490 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:35.945501 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:35.945505 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:35.945547 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.945607 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.945625 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 06:56:35.945938 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.940815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:33579 (local address 127.0.191.195:34855)
0926 06:56:35.941036 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:35.941070 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:35.941635 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:35.941869 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:35.941872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:35.941880 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:35.942378 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.942382 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.943035 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.943036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.943958 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.943972 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.944477 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.944478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.944650 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.945308 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:56:35.945309 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:35.945350 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:56:35.945404 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:35.945557 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:35.945560 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:35.945562 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:56:35.945611 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 06:56:35.945687 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260926 06:56:35.946375 19828 heartbeater.cc:347] Connected to a master server at 127.0.191.254:33579
I20260926 06:56:35.946475 19828 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:35.946663 19828 heartbeater.cc:511] Master 127.0.191.254:33579 requested a full tablet report, sending...
I20260926 06:56:35.947006 19378 ts_manager.cc:194] Registered new tserver with Master: d54012d20d55400c8c9e4828efa46ebe (127.0.191.195:45827)
I20260926 06:56:35.947361 19378 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.191.195:34855
I20260926 06:56:35.955562 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:56:35.959795 19417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.956883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46104 (local address 127.0.191.254:33579)
0926 06:56:35.956925 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.956927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.957029 (+ 102us) server_negotiation.cc:407] Connection header received
0926 06:56:35.957103 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.957103 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.957112 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.957156 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:35.957329 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.957834 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.958593 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.958796 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.958869 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.959512 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:35.959513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:35.959514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:35.959518 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:35.959530 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:35.959535 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:35.959641 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.959706 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.959715 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 06:56:35.961184 19378 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:56:35.962725 19378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:56:35.969040 19838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.964986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44802 (local address 127.0.191.194:42811)
0926 06:56:35.965246 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.965251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.965261 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:56:35.965296 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.965298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.965314 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.965442 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:35.965677 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.966813 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.968037 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.968638 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.968720 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.968820 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.968916 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.968954 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":90,"threads_started":1}
I20260926 06:56:35.969046 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.964897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:42811 (local address 127.0.0.1:44802)
0926 06:56:35.965065 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:35.965109 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:35.965174 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:35.965463 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:35.965466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:35.965468 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:35.965592 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.965595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.966909 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.966910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.967959 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.967961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.968730 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.968731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.968808 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.968826 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 06:56:35.968901 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:56:35.968958 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":59,"threads_started":1}
I20260926 06:56:35.969569 19840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.965210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56526 (local address 127.0.191.195:45827)
0926 06:56:35.965388 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.965392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.965619 (+ 227us) server_negotiation.cc:407] Connection header received
0926 06:56:35.965664 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.965666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.965680 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.965744 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:35.965925 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.966998 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.968378 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.969092 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.969162 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.969244 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.969454 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.969488 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260926 06:56:35.969580 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.965244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60804 (local address 127.0.191.193:39753)
0926 06:56:35.965451 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:35.965456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:35.965466 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:56:35.965593 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:35.965596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:35.965612 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:35.965684 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:35.965874 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.966715 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.968187 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.968778 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.968842 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.968930 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:35.969453 (+ 523us) server_negotiation.cc:299] Negotiation successful
0926 06:56:35.969503 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260926 06:56:35.969830 19631 tablet_service.cc:1467] Processing CreateTablet for tablet 359dc3192cf147d9aad283ead1f332db (DEFAULT_TABLE table=test-table [id=889058011d6d49b9bfa91ce6b5d3459e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:35.970043 19836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.965108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:45827 (local address 127.0.0.1:56526)
0926 06:56:35.965509 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:35.965517 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:35.965565 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:35.965801 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:35.965802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:35.965804 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:35.965836 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.965838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.967066 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.967067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.968290 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.968291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.969278 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.969279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.969341 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.969346 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:56:35.969437 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 06:56:35.969944 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":64,"threads_started":1}
I20260926 06:56:35.970232 19631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 359dc3192cf147d9aad283ead1f332db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:35.970280 19839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:35.964912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39753 (local address 127.0.0.1:60804)
0926 06:56:35.965414 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:35.965425 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:35.965480 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:35.965751 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:35.965751 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:35.965753 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:35.965796 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.965797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.966798 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.966799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.968081 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:35.968083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:35.968962 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:35.968962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:35.969018 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:35.969022 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:56:35.969069 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 06:56:35.969935 (+ 866us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":63,"thread_start_us":97,"threads_started":1}
I20260926 06:56:35.970795 19762 tablet_service.cc:1467] Processing CreateTablet for tablet 359dc3192cf147d9aad283ead1f332db (DEFAULT_TABLE table=test-table [id=889058011d6d49b9bfa91ce6b5d3459e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:35.970867 19500 tablet_service.cc:1467] Processing CreateTablet for tablet 359dc3192cf147d9aad283ead1f332db (DEFAULT_TABLE table=test-table [id=889058011d6d49b9bfa91ce6b5d3459e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:35.971120 19762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 359dc3192cf147d9aad283ead1f332db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:35.971216 19500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 359dc3192cf147d9aad283ead1f332db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:35.974296 19843 tablet_bootstrap.cc:492] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac: Bootstrap starting.
I20260926 06:56:35.974675 19844 tablet_bootstrap.cc:492] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe: Bootstrap starting.
I20260926 06:56:35.975008 19843 tablet_bootstrap.cc:654] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:35.975291 19844 tablet_bootstrap.cc:654] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:35.975309 19843 log.cc:824] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:35.975600 19844 log.cc:824] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:35.975634 19842 tablet_bootstrap.cc:492] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824: Bootstrap starting.
I20260926 06:56:35.976018 19843 tablet_bootstrap.cc:492] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac: No bootstrap required, opened a new log
I20260926 06:56:35.976109 19843 ts_tablet_manager.cc:1397] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:56:35.976289 19842 tablet_bootstrap.cc:654] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:35.976410 19844 tablet_bootstrap.cc:492] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe: No bootstrap required, opened a new log
I20260926 06:56:35.976487 19844 ts_tablet_manager.cc:1397] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:56:35.976740 19842 log.cc:824] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:35.977641 19842 tablet_bootstrap.cc:492] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824: No bootstrap required, opened a new log
I20260926 06:56:35.977728 19842 ts_tablet_manager.cc:1397] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:56:35.978760 19843 raft_consensus.cc:348] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "590b099edfc7430f92838e16ee843824" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42811 } } peers { permanent_uuid: "4d43deb998a14c5e97a90ac68abff7ac" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39753 } } peers { permanent_uuid: "d54012d20d55400c8c9e4828efa46ebe" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45827 } }
I20260926 06:56:35.978938 19843 raft_consensus.cc:374] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:35.978976 19843 raft_consensus.cc:729] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d43deb998a14c5e97a90ac68abff7ac, State: Initialized, Role: FOLLOWER
I20260926 06:56:35.979024 19844 raft_consensus.cc:348] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "590b099edfc7430f92838e16ee843824" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42811 } } peers { permanent_uuid: "4d43deb998a14c5e97a90ac68abff7ac" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39753 } } peers { permanent_uuid: "d54012d20d55400c8c9e4828efa46ebe" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45827 } }
I20260926 06:56:35.979130 19843 consensus_queue.cc:260] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "590b099edfc7430f92838e16ee843824" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42811 } } peers { permanent_uuid: "4d43deb998a14c5e97a90ac68abff7ac" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39753 } } peers { permanent_uuid: "d54012d20d55400c8c9e4828efa46ebe" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45827 } }
I20260926 06:56:35.979172 19844 raft_consensus.cc:374] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:35.979205 19844 raft_consensus.cc:729] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d54012d20d55400c8c9e4828efa46ebe, State: Initialized, Role: FOLLOWER
I20260926 06:56:35.979336 19844 consensus_queue.cc:260] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "590b099edfc7430f92838e16ee843824" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42811 } } peers { permanent_uuid: "4d43deb998a14c5e97a90ac68abff7ac" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39753 } } peers { permanent_uuid: "d54012d20d55400c8c9e4828efa46ebe" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45827 } }
I20260926 06:56:35.979390 19843 ts_tablet_manager.cc:1428] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260926 06:56:35.979502 19566 heartbeater.cc:503] Master 127.0.191.254:33579 was elected leader, sending a full tablet report...
I20260926 06:56:35.979586 19844 ts_tablet_manager.cc:1428] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:56:35.979743 19828 heartbeater.cc:503] Master 127.0.191.254:33579 was elected leader, sending a full tablet report...
I20260926 06:56:35.980065 19842 raft_consensus.cc:348] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "590b099edfc7430f92838e16ee843824" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42811 } } peers { permanent_uuid: "4d43deb998a14c5e97a90ac68abff7ac" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39753 } } peers { permanent_uuid: "d54012d20d55400c8c9e4828efa46ebe" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45827 } }
I20260926 06:56:35.980223 19842 raft_consensus.cc:374] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:35.980261 19842 raft_consensus.cc:729] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 590b099edfc7430f92838e16ee843824, State: Initialized, Role: FOLLOWER
I20260926 06:56:35.980400 19842 consensus_queue.cc:260] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "590b099edfc7430f92838e16ee843824" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42811 } } peers { permanent_uuid: "4d43deb998a14c5e97a90ac68abff7ac" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39753 } } peers { permanent_uuid: "d54012d20d55400c8c9e4828efa46ebe" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45827 } }
I20260926 06:56:35.980659 19842 ts_tablet_manager.cc:1428] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:56:35.980703 19697 heartbeater.cc:503] Master 127.0.191.254:33579 was elected leader, sending a full tablet report...
I20260926 06:56:36.012910 19850 raft_consensus.cc:482] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:56:36.013025 19850 raft_consensus.cc:504] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "590b099edfc7430f92838e16ee843824" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42811 } } peers { permanent_uuid: "4d43deb998a14c5e97a90ac68abff7ac" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39753 } } peers { permanent_uuid: "d54012d20d55400c8c9e4828efa46ebe" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45827 } }
I20260926 06:56:36.013365 19850 leader_election.cc:288] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d43deb998a14c5e97a90ac68abff7ac (127.0.191.193:39753), d54012d20d55400c8c9e4828efa46ebe (127.0.191.195:45827)
I20260926 06:56:36.017204 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:36.013686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:53315 (local address 127.0.191.193:39753)
0926 06:56:36.013758 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:36.013760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:36.013770 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:56:36.013802 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:36.013803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:36.013813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:36.013869 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:36.014072 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:36.014814 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:36.016249 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:36.016796 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:36.016876 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:36.016884 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:36.017089 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 06:56:36.017134 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 06:56:36.017223 19840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:36.013621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:42317 (local address 127.0.191.195:45827)
0926 06:56:36.013665 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:36.013667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:36.014007 (+ 340us) server_negotiation.cc:407] Connection header received
0926 06:56:36.014061 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:36.014062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:36.014072 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:36.014126 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:36.014252 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:36.014926 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:36.016249 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:36.016796 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:36.016876 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:36.016884 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:36.017124 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 06:56:36.017162 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:56:36.017216 19699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:36.013538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39753 (local address 127.0.191.194:53315)
0926 06:56:36.013597 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:36.013609 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:36.013649 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:36.013896 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:36.013897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:36.013899 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:36.013979 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:36.013981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:36.014886 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:36.014886 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:36.016155 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:36.016158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:36.016905 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:36.016906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:36.016996 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:36.017000 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:56:36.017065 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 06:56:36.017118 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":29}
I20260926 06:56:36.017223 19851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:36.013539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:45827 (local address 127.0.191.194:42317)
0926 06:56:36.013918 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:36.013929 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:36.013999 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:36.014142 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:36.014143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:36.014144 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:36.014195 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:36.014196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:36.014994 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:36.014995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:36.016160 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:36.016162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:36.016907 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:36.016907 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:36.016996 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:36.017000 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:56:36.017114 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 06:56:36.017145 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":160,"thread_start_us":125,"threads_started":1}
I20260926 06:56:36.017692 19520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359dc3192cf147d9aad283ead1f332db" candidate_uuid: "590b099edfc7430f92838e16ee843824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d43deb998a14c5e97a90ac68abff7ac" is_pre_election: true
I20260926 06:56:36.017875 19520 raft_consensus.cc:2454] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 590b099edfc7430f92838e16ee843824 in term 0.
I20260926 06:56:36.017992 19782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359dc3192cf147d9aad283ead1f332db" candidate_uuid: "590b099edfc7430f92838e16ee843824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d54012d20d55400c8c9e4828efa46ebe" is_pre_election: true
I20260926 06:56:36.018157 19782 raft_consensus.cc:2454] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 590b099edfc7430f92838e16ee843824 in term 0.
I20260926 06:56:36.018155 19585 leader_election.cc:302] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d43deb998a14c5e97a90ac68abff7ac, 590b099edfc7430f92838e16ee843824; no voters:
I20260926 06:56:36.018278 19850 raft_consensus.cc:2781] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:56:36.018352 19850 raft_consensus.cc:482] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:56:36.018368 19850 raft_consensus.cc:3037] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:36.019235 19850 raft_consensus.cc:504] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "590b099edfc7430f92838e16ee843824" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42811 } } peers { permanent_uuid: "4d43deb998a14c5e97a90ac68abff7ac" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39753 } } peers { permanent_uuid: "d54012d20d55400c8c9e4828efa46ebe" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45827 } }
I20260926 06:56:36.019403 19850 leader_election.cc:288] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [CANDIDATE]: Term 1 election: Requested vote from peers 4d43deb998a14c5e97a90ac68abff7ac (127.0.191.193:39753), d54012d20d55400c8c9e4828efa46ebe (127.0.191.195:45827)
I20260926 06:56:36.019589 19520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359dc3192cf147d9aad283ead1f332db" candidate_uuid: "590b099edfc7430f92838e16ee843824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d43deb998a14c5e97a90ac68abff7ac"
I20260926 06:56:36.019589 19782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359dc3192cf147d9aad283ead1f332db" candidate_uuid: "590b099edfc7430f92838e16ee843824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d54012d20d55400c8c9e4828efa46ebe"
I20260926 06:56:36.019666 19782 raft_consensus.cc:3037] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:36.019667 19520 raft_consensus.cc:3037] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:36.020602 19520 raft_consensus.cc:2454] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 590b099edfc7430f92838e16ee843824 in term 1.
I20260926 06:56:36.020602 19782 raft_consensus.cc:2454] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 590b099edfc7430f92838e16ee843824 in term 1.
I20260926 06:56:36.020763 19585 leader_election.cc:302] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d43deb998a14c5e97a90ac68abff7ac, 590b099edfc7430f92838e16ee843824; no voters:
I20260926 06:56:36.020906 19850 raft_consensus.cc:2781] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:36.021059 19850 raft_consensus.cc:686] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [term 1 LEADER]: Becoming Leader. State: Replica: 590b099edfc7430f92838e16ee843824, State: Running, Role: LEADER
I20260926 06:56:36.021157 19850 consensus_queue.cc:237] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "590b099edfc7430f92838e16ee843824" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42811 } } peers { permanent_uuid: "4d43deb998a14c5e97a90ac68abff7ac" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39753 } } peers { permanent_uuid: "d54012d20d55400c8c9e4828efa46ebe" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45827 } }
I20260926 06:56:36.021939 19377 catalog_manager.cc:5495] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 reported cstate change: term changed from 0 to 1, leader changed from <none> to 590b099edfc7430f92838e16ee843824 (127.0.191.194). New cstate: current_term: 1 leader_uuid: "590b099edfc7430f92838e16ee843824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "590b099edfc7430f92838e16ee843824" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 42811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d43deb998a14c5e97a90ac68abff7ac" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d54012d20d55400c8c9e4828efa46ebe" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45827 } health_report { overall_health: UNKNOWN } } }
I20260926 06:56:36.028370 19838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:36.025945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44806 (local address 127.0.191.194:42811)
0926 06:56:36.026005 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:36.026007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:36.026016 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:56:36.026072 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:36.026073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:36.026091 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:36.026178 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:56:36.026325 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:36.026798 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:36.027576 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:36.027778 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:36.027867 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:36.028030 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:56:36.028123 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:56:36.028232 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:36.028264 (+ 32us) server_negotiation.cc:299] Negotiation successful
0926 06:56:36.028271 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:56:36.030190 19782 raft_consensus.cc:1261] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe [term 1 FOLLOWER]: Refusing update from remote peer 590b099edfc7430f92838e16ee843824: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:56:36.030190 19520 raft_consensus.cc:1261] T 359dc3192cf147d9aad283ead1f332db P 4d43deb998a14c5e97a90ac68abff7ac [term 1 FOLLOWER]: Refusing update from remote peer 590b099edfc7430f92838e16ee843824: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:56:36.030593 19852 consensus_queue.cc:1035] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d43deb998a14c5e97a90ac68abff7ac" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 39753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:56:36.030730 19850 consensus_queue.cc:1035] T 359dc3192cf147d9aad283ead1f332db P 590b099edfc7430f92838e16ee843824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d54012d20d55400c8c9e4828efa46ebe" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 45827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:56:36.033288 19856 mvcc.cc:205] Tried to move back new op lower bound from 7333502140537004032 to 7333502140503339008. Current Snapshot: MvccSnapshot[applied={T|T < 7333502140537004032}]
I20260926 06:56:36.033486 19857 mvcc.cc:205] Tried to move back new op lower bound from 7333502140537004032 to 7333502140503339008. Current Snapshot: MvccSnapshot[applied={T|T < 7333502140537004032}]
I20260926 06:56:36.035146 19377 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:56:36.035224 19377 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:56:36.035938 19377 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41e20a484d1f47609688b068ee6385f3: Sending DeleteTablet for 3 replicas of tablet 359dc3192cf147d9aad283ead1f332db
I20260926 06:56:36.036298 19631 tablet_service.cc:1514] Processing DeleteTablet for tablet 359dc3192cf147d9aad283ead1f332db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:36 UTC) from {username='slave'} at 127.0.0.1:44802
I20260926 06:56:36.036324 19500 tablet_service.cc:1514] Processing DeleteTablet for tablet 359dc3192cf147d9aad283ead1f332db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:36 UTC) from {username='slave'} at 127.0.0.1:60804
I20260926 06:56:36.036556 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 19438
I20260926 06:56:36.036564 19762 tablet_service.cc:1514] Processing DeleteTablet for tablet 359dc3192cf147d9aad283ead1f332db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:36 UTC) from {username='slave'} at 127.0.0.1:56526
I20260926 06:56:36.036796 19866 tablet_replica.cc:330] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe: stopping tablet replica
I20260926 06:56:36.036899 19866 raft_consensus.cc:2229] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:56:36.037001 19866 raft_consensus.cc:2258] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:56:36.037443 19866 ts_tablet_manager.cc:1905] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:56:36.039427 19866 ts_tablet_manager.cc:1918] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:56:36.039505 19866 log.cc:1196] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/359dc3192cf147d9aad283ead1f332db
I20260926 06:56:36.039778 19866 ts_tablet_manager.cc:1939] T 359dc3192cf147d9aad283ead1f332db P d54012d20d55400c8c9e4828efa46ebe: Deleting consensus metadata
I20260926 06:56:36.040350 19365 catalog_manager.cc:4841] TS d54012d20d55400c8c9e4828efa46ebe (127.0.191.195:45827): tablet 359dc3192cf147d9aad283ead1f332db (table test-table [id=889058011d6d49b9bfa91ce6b5d3459e]) successfully deleted
W20260926 06:56:36.043087 19363 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:39753 (error 108)
I20260926 06:56:36.043582 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 19569
I20260926 06:56:36.043711 19839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:36.043567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:39753 (local address 127.0.0.1:60820)
0926 06:56:36.043630 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:36.043674 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:39753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260926 06:56:36.043816 19363 catalog_manager.cc:4584] TS 4d43deb998a14c5e97a90ac68abff7ac (127.0.191.193:39753): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 359dc3192cf147d9aad283ead1f332db: Network error: Client connection negotiation failed: client connection to 127.0.191.193:39753: connect: Connection refused (error 111)
I20260926 06:56:36.048718 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 19700
I20260926 06:56:36.048764 19839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:36.048649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:42811 (local address 127.0.0.1:44810)
0926 06:56:36.048705 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:36.048731 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.194:42811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 06:56:36.053972 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 19347
2026-09-26T06:56:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (659 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:36 dist-test-slave-ws74 krb5kdc[19870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:36 dist-test-slave-ws74 krb5kdc[19870](info): set up 2 sockets
Sep 26 06:56:36 dist-test-slave-ws74 krb5kdc[19870](info): commencing operation
krb5kdc: starting...
W20260926 06:56:37.086226 767 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:37 dist-test-slave-ws74 krb5kdc[19870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:56:37.204460 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:41329
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:41329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:37.283360 19883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:37.283555 19883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:37.283576 19883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:37.284646 19883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:37.284680 19883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:37.284694 19883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:37.284706 19883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:37.284718 19883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:37.286113 19883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:41329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:41329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:37.286302 19883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:37.286468 19883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:37.291214 19889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:37.291209 19883 server_base.cc:1034] running on GCE node
W20260926 06:56:37.291193 19891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:37.291185 19888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:37.291631 19883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:37.291888 19883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:37.293075 19883 hybrid_clock.cc:648] HybridClock initialized: now 1790405797293061 us; error 29 us; skew 500 ppm
Sep 26 06:56:37 dist-test-slave-ws74 krb5kdc[19870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405797, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:37.295483 19883 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:56:37.295926 19883 webserver.cc:460] Webserver started at http://127.0.191.254:34671/ using document root <none> and password file <none>
I20260926 06:56:37.296203 19883 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:37.296259 19883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:37.296351 19883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:37.297320 19883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "01939f4def7c4f9b907fab587784e2cb"
format_stamp: "Formatted at 2026-09-26 06:56:37 on dist-test-slave-ws74"
server_key: "f0069c74ab6eb8a17ae74fe1d8c77db4"
server_key_iv: "67c9a0d9d266f927cd2e8e69e9a32678"
server_key_version: "encryptionkey@0"
I20260926 06:56:37.297684 19883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "01939f4def7c4f9b907fab587784e2cb"
format_stamp: "Formatted at 2026-09-26 06:56:37 on dist-test-slave-ws74"
server_key: "f0069c74ab6eb8a17ae74fe1d8c77db4"
server_key_iv: "67c9a0d9d266f927cd2e8e69e9a32678"
server_key_version: "encryptionkey@0"
I20260926 06:56:37.298934 19883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:37.299578 19898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:37.299737 19883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:37.299839 19883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "01939f4def7c4f9b907fab587784e2cb"
format_stamp: "Formatted at 2026-09-26 06:56:37 on dist-test-slave-ws74"
server_key: "f0069c74ab6eb8a17ae74fe1d8c77db4"
server_key_iv: "67c9a0d9d266f927cd2e8e69e9a32678"
server_key_version: "encryptionkey@0"
I20260926 06:56:37.299928 19883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:37.329212 19883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:37.331099 19883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:37.331264 19883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:37.335337 19883 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:41329
I20260926 06:56:37.335371 19950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:41329 every 8 connection(s)
I20260926 06:56:37.335773 19883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:37.336598 19951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:37.338893 19951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb: Bootstrap starting.
I20260926 06:56:37.339171 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 19883
I20260926 06:56:37.339296 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:37.339507 767 external_mini_cluster.cc:1428] Setting key da2cb65e8144928b50cd65cbf2ed579e
I20260926 06:56:37.339552 19951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:37.339851 19951 log.cc:824] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:37.340553 19951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb: No bootstrap required, opened a new log
I20260926 06:56:37.342392 19951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01939f4def7c4f9b907fab587784e2cb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 41329 } }
I20260926 06:56:37.342523 19951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:37.342546 19951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01939f4def7c4f9b907fab587784e2cb, State: Initialized, Role: FOLLOWER
I20260926 06:56:37.342679 19951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01939f4def7c4f9b907fab587784e2cb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 41329 } }
I20260926 06:56:37.342751 19951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:37.342801 19951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:37.342849 19951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:37.343394 19951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01939f4def7c4f9b907fab587784e2cb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 41329 } }
I20260926 06:56:37.343509 19951 leader_election.cc:302] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01939f4def7c4f9b907fab587784e2cb; no voters:
I20260926 06:56:37.343680 19951 leader_election.cc:288] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:37.343729 19956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:37.343863 19956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [term 1 LEADER]: Becoming Leader. State: Replica: 01939f4def7c4f9b907fab587784e2cb, State: Running, Role: LEADER
I20260926 06:56:37.343936 19951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:37.343948 19956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01939f4def7c4f9b907fab587784e2cb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 41329 } }
I20260926 06:56:37.344298 19956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01939f4def7c4f9b907fab587784e2cb. Latest consensus state: current_term: 1 leader_uuid: "01939f4def7c4f9b907fab587784e2cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01939f4def7c4f9b907fab587784e2cb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 41329 } } }
I20260926 06:56:37.344376 19956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:37.344554 19957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01939f4def7c4f9b907fab587784e2cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01939f4def7c4f9b907fab587784e2cb" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 41329 } } }
I20260926 06:56:37.344594 19962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:37.346288 19962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 06:56:37 dist-test-slave-ws74 krb5kdc[19870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:37.346855 19957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:37.348042 19962 catalog_manager.cc:1347] Generated new cluster ID: 0ada66e8ab334a4b8ad6774677b3781f
I20260926 06:56:37.348091 19962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:37.356097 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.340026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43558 (local address 127.0.191.254:41329)
0926 06:56:37.340230 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:37.340240 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:37.340271 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:56:37.340745 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:37.340767 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:37.340814 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:37.340982 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:37.341673 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.342566 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.343450 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.344113 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.344246 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.347150 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:37.347155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:37.347155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:37.347168 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:37.354569 (+ 7401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:37.354938 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:37.354942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:37.354942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:37.355045 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:37.355266 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:37.355267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:37.355267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:37.355505 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:37.355676 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:37.355774 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 06:56:37.355909 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260926 06:56:37.384176 19962 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:37.384924 19962 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:37.398573 19962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb: Generated new TSK 0
I20260926 06:56:37.398794 19962 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.191.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
Entry for principal kudu/127.0.191.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab.
I20260926 06:56:37.424989 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:41329
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:37.505342 19977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:37.505576 19977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:37.505606 19977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:37.506775 19977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:37.506817 19977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:37.506843 19977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:56:37.508312 19977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.193.keytab
--principal=kudu/127.0.191.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:41329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:37.508562 19977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:37.508802 19977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:37.514202 19983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:37.514376 19985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:37.514410 19977 server_base.cc:1034] running on GCE node
W20260926 06:56:37.514225 19982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:37.514659 19977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:37.514853 19977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:37.516016 19977 hybrid_clock.cc:648] HybridClock initialized: now 1790405797516004 us; error 42 us; skew 500 ppm
Sep 26 06:56:37 dist-test-slave-ws74 krb5kdc[19870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405797, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:37.518623 19977 init.cc:377] Logged in from keytab as kudu/127.0.191.193@KRBTEST.COM (short username kudu)
I20260926 06:56:37.519081 19977 webserver.cc:460] Webserver started at http://127.0.191.193:39199/ using document root <none> and password file <none>
I20260926 06:56:37.519325 19977 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:37.519371 19977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:37.519528 19977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:37.520556 19977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "ee3fec0560d148ebaf9370d499a64b4e"
format_stamp: "Formatted at 2026-09-26 06:56:37 on dist-test-slave-ws74"
server_key: "b36524447efe64f0df147d61d7d0f18c"
server_key_iv: "9a4b0e00ad389e2b22f5cf664d06b577"
server_key_version: "encryptionkey@0"
I20260926 06:56:37.520905 19977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "ee3fec0560d148ebaf9370d499a64b4e"
format_stamp: "Formatted at 2026-09-26 06:56:37 on dist-test-slave-ws74"
server_key: "b36524447efe64f0df147d61d7d0f18c"
server_key_iv: "9a4b0e00ad389e2b22f5cf664d06b577"
server_key_version: "encryptionkey@0"
I20260926 06:56:37.522313 19977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:56:37.523061 19992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:37.523231 19977 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:37.523337 19977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "ee3fec0560d148ebaf9370d499a64b4e"
format_stamp: "Formatted at 2026-09-26 06:56:37 on dist-test-slave-ws74"
server_key: "b36524447efe64f0df147d61d7d0f18c"
server_key_iv: "9a4b0e00ad389e2b22f5cf664d06b577"
server_key_version: "encryptionkey@0"
I20260926 06:56:37.523411 19977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:37.542415 19977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:37.544812 19977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:37.544971 19977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:37.545195 19977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:37.545554 19977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:37.545615 19977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:37.545652 19977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:37.545680 19977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:37.551704 19977 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:38611
I20260926 06:56:37.551753 20105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:38611 every 8 connection(s)
I20260926 06:56:37.552059 19977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:56:37 dist-test-slave-ws74 krb5kdc[19870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405797, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.193@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:37.559965 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 19977
I20260926 06:56:37.560087 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:56:37.560328 767 external_mini_cluster.cc:1428] Setting key 994f0e6e54d44edaf53e574bfdfadba6
I20260926 06:56:37.561811 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.552746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:41329 (local address 127.0.191.193:50917)
0926 06:56:37.552940 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:37.552982 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:37.554028 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:37.554247 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:37.554251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:37.554523 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:37.554710 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:37.554714 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.555389 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.555390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:37.556194 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:37.556196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.556529 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.556529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:37.556630 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.557097 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:56:37.557105 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:37.558515 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:37.560254 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:37.560259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:37.560261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:37.560568 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:37.560836 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:37.560837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:37.560837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:37.560937 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:37.561236 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:37.561241 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:37.561444 (+ 203us) client_negotiation.cc:769] Sending connection context
0926 06:56:37.561534 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 06:56:37.561652 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":124,"threads_started":1}
I20260926 06:56:37.561818 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.552784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:50917 (local address 127.0.191.254:41329)
0926 06:56:37.552847 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:37.552849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:37.553079 (+ 230us) server_negotiation.cc:407] Connection header received
0926 06:56:37.554123 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:37.554125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:37.554135 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:37.554193 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:37.554784 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.555280 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.556268 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.556454 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.556511 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.558601 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:37.558603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:37.558603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:37.558610 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:37.560172 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:37.560704 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:37.560705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:37.560706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:37.560767 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:37.561020 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:37.561021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:37.561021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:37.561167 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:37.561245 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:37.561576 (+ 331us) server_negotiation.cc:299] Negotiation successful
0926 06:56:37.561673 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 06:56:37.562403 20106 heartbeater.cc:347] Connected to a master server at 127.0.191.254:41329
I20260926 06:56:37.562520 20106 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:37.562700 20106 heartbeater.cc:511] Master 127.0.191.254:41329 requested a full tablet report, sending...
I20260926 06:56:37.563253 19915 ts_manager.cc:194] Registered new tserver with Master: ee3fec0560d148ebaf9370d499a64b4e (127.0.191.193:38611)
I20260926 06:56:37.563880 19915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.193@KRBTEST.COM'} at 127.0.191.193:50917
WARNING: no policy specified for kudu/127.0.191.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
Entry for principal kudu/127.0.191.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab.
I20260926 06:56:37.585894 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:41329
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:37.666575 20111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:37.666775 20111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:37.666798 20111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:37.667869 20111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:37.667905 20111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:37.667932 20111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:56:37.669283 20111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.194.keytab
--principal=kudu/127.0.191.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:41329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:37.669528 20111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:37.669708 20111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:37.674454 20117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:37.674635 20111 server_base.cc:1034] running on GCE node
W20260926 06:56:37.674460 20116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:37.674454 20119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:37.674948 20111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:37.675189 20111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:37.676347 20111 hybrid_clock.cc:648] HybridClock initialized: now 1790405797676341 us; error 39 us; skew 500 ppm
Sep 26 06:56:37 dist-test-slave-ws74 krb5kdc[19870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405797, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:37.678860 20111 init.cc:377] Logged in from keytab as kudu/127.0.191.194@KRBTEST.COM (short username kudu)
I20260926 06:56:37.679229 20111 webserver.cc:460] Webserver started at http://127.0.191.194:40957/ using document root <none> and password file <none>
I20260926 06:56:37.679397 20111 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:37.679436 20111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:37.679522 20111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:37.680447 20111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "1f20f72c2de34d639f0440bd47351369"
format_stamp: "Formatted at 2026-09-26 06:56:37 on dist-test-slave-ws74"
server_key: "3d5833a9a5121bf8fb00d361ec1f108f"
server_key_iv: "63ade2a3ef219d182cf4eb96d4aafa80"
server_key_version: "encryptionkey@0"
I20260926 06:56:37.680735 20111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "1f20f72c2de34d639f0440bd47351369"
format_stamp: "Formatted at 2026-09-26 06:56:37 on dist-test-slave-ws74"
server_key: "3d5833a9a5121bf8fb00d361ec1f108f"
server_key_iv: "63ade2a3ef219d182cf4eb96d4aafa80"
server_key_version: "encryptionkey@0"
I20260926 06:56:37.682013 20111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:56:37.682638 20126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:37.682833 20111 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:37.682941 20111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "1f20f72c2de34d639f0440bd47351369"
format_stamp: "Formatted at 2026-09-26 06:56:37 on dist-test-slave-ws74"
server_key: "3d5833a9a5121bf8fb00d361ec1f108f"
server_key_iv: "63ade2a3ef219d182cf4eb96d4aafa80"
server_key_version: "encryptionkey@0"
I20260926 06:56:37.683012 20111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:37.718634 20111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:37.720683 20111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:37.720837 20111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:37.721081 20111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:37.721448 20111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:37.721485 20111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:37.721525 20111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:37.721561 20111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:37.727346 20111 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:46119
I20260926 06:56:37.727437 20239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:46119 every 8 connection(s)
I20260926 06:56:37.727717 20111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:56:37.730749 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 20111
I20260926 06:56:37.730895 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:56:37.731107 767 external_mini_cluster.cc:1428] Setting key 177219838f3831d2d12af94bc6353aa5
Sep 26 06:56:37 dist-test-slave-ws74 krb5kdc[19870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405797, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.194@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:37.737748 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.728454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:50311 (local address 127.0.191.254:41329)
0926 06:56:37.728536 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:37.728539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:37.728725 (+ 186us) server_negotiation.cc:407] Connection header received
0926 06:56:37.729700 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:37.729702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:37.729714 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:37.729767 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:37.730388 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.730983 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.732204 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.732409 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.732495 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.734850 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:37.734852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:37.734852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:37.734860 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:37.736283 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:37.736736 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:37.736738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:37.736738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:37.736798 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:37.737057 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:37.737058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:37.737058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:37.737167 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:37.737241 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:37.737579 (+ 338us) server_negotiation.cc:299] Negotiation successful
0926 06:56:37.737657 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 06:56:37.737983 20242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.728419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:41329 (local address 127.0.191.194:50311)
0926 06:56:37.728592 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:37.728639 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:37.729590 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:37.729838 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:37.729842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:37.730107 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:37.730292 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:37.730296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.731070 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.731071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:37.732118 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:37.732120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.732510 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.732511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:37.732640 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.733178 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:56:37.733188 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:37.734751 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:37.736430 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:37.736436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:37.736438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:37.736659 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:37.736873 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:37.736874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:37.736874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:37.736983 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:37.737244 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:37.737250 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:37.737427 (+ 177us) client_negotiation.cc:769] Sending connection context
0926 06:56:37.737556 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 06:56:37.737708 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260926 06:56:37.738530 20240 heartbeater.cc:347] Connected to a master server at 127.0.191.254:41329
I20260926 06:56:37.738636 20240 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:37.738804 20240 heartbeater.cc:511] Master 127.0.191.254:41329 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.191.195@KRBTEST.COM; defaulting to no policy
I20260926 06:56:37.739184 19915 ts_manager.cc:194] Registered new tserver with Master: 1f20f72c2de34d639f0440bd47351369 (127.0.191.194:46119)
I20260926 06:56:37.739642 19915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.194@KRBTEST.COM'} at 127.0.191.194:50311
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
Entry for principal kudu/127.0.191.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab.
I20260926 06:56:37.756075 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:41329
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:37.835152 20245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:37.835387 20245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:37.835424 20245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:37.836589 20245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:37.836655 20245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:37.836699 20245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:56:37.838243 20245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/krb5kdc/kudu_127.0.191.195.keytab
--principal=kudu/127.0.191.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:41329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:37.838518 20245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:37.838743 20245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:37.843573 20250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:37.843694 20253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:37.843852 20245 server_base.cc:1034] running on GCE node
W20260926 06:56:37.843879 20251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:37.844168 20245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:37.844448 20245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:37.845609 20245 hybrid_clock.cc:648] HybridClock initialized: now 1790405797845600 us; error 39 us; skew 500 ppm
Sep 26 06:56:37 dist-test-slave-ws74 krb5kdc[19870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405797, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:37.848268 20245 init.cc:377] Logged in from keytab as kudu/127.0.191.195@KRBTEST.COM (short username kudu)
I20260926 06:56:37.848766 20245 webserver.cc:460] Webserver started at http://127.0.191.195:42259/ using document root <none> and password file <none>
I20260926 06:56:37.849001 20245 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:37.849069 20245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:37.849225 20245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:37.850327 20245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "f90ac71f14084b49aa9a88ecf88c9dee"
format_stamp: "Formatted at 2026-09-26 06:56:37 on dist-test-slave-ws74"
server_key: "2514a592ab6bc1c49f72db8cedb434b2"
server_key_iv: "3e8c9a73006782c3bd145ae8cb792907"
server_key_version: "encryptionkey@0"
I20260926 06:56:37.850695 20245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "f90ac71f14084b49aa9a88ecf88c9dee"
format_stamp: "Formatted at 2026-09-26 06:56:37 on dist-test-slave-ws74"
server_key: "2514a592ab6bc1c49f72db8cedb434b2"
server_key_iv: "3e8c9a73006782c3bd145ae8cb792907"
server_key_version: "encryptionkey@0"
I20260926 06:56:37.851994 20245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:56:37.852691 20260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:37.852883 20245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:37.852978 20245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "f90ac71f14084b49aa9a88ecf88c9dee"
format_stamp: "Formatted at 2026-09-26 06:56:37 on dist-test-slave-ws74"
server_key: "2514a592ab6bc1c49f72db8cedb434b2"
server_key_iv: "3e8c9a73006782c3bd145ae8cb792907"
server_key_version: "encryptionkey@0"
I20260926 06:56:37.853065 20245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:37.861109 20245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:37.863379 20245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:37.863538 20245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:37.863749 20245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:37.864048 20245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:37.864100 20245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:37.864140 20245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:37.864167 20245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:37.870085 20245 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:43017
I20260926 06:56:37.870131 20373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:43017 every 8 connection(s)
I20260926 06:56:37.870447 20245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:56:37 dist-test-slave-ws74 krb5kdc[19870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405797, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.195@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:37.879863 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.871227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:36357 (local address 127.0.191.254:41329)
0926 06:56:37.871275 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:37.871276 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:37.871485 (+ 209us) server_negotiation.cc:407] Connection header received
0926 06:56:37.872456 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:37.872457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:37.872470 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:37.872533 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:37.873185 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.873765 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.874781 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.874982 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.875070 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.877283 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:37.877285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:37.877285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:37.877293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:37.878581 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:37.878949 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:37.878949 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:37.878950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:37.879008 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:37.879240 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:37.879242 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:37.879242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:37.879386 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:37.879470 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:37.879726 (+ 256us) server_negotiation.cc:299] Negotiation successful
0926 06:56:37.879792 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:56:37.879958 20376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.871177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:41329 (local address 127.0.191.195:36357)
0926 06:56:37.871357 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:37.871400 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:37.872332 (+ 932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:37.872606 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:37.872610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:37.872872 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:37.873071 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:37.873075 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.873858 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.873859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:37.874673 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:37.874675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.875079 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.875081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:37.875211 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.875737 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:56:37.875746 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:37.877191 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:37.878657 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:37.878663 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:37.878664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:37.878881 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:37.879066 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:56:37.879066 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:56:37.879067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:56:37.879159 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:56:37.879452 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:37.879474 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:37.879609 (+ 135us) client_negotiation.cc:769] Sending connection context
0926 06:56:37.879692 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 06:56:37.879806 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260926 06:56:37.880559 20374 heartbeater.cc:347] Connected to a master server at 127.0.191.254:41329
I20260926 06:56:37.880678 20374 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:37.880695 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 20245
I20260926 06:56:37.880800 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:56:37.880916 20374 heartbeater.cc:511] Master 127.0.191.254:41329 requested a full tablet report, sending...
I20260926 06:56:37.881014 767 external_mini_cluster.cc:1428] Setting key 0f3e8fb88141ebeeb558f1a6c79e1e98
I20260926 06:56:37.881307 19915 ts_manager.cc:194] Registered new tserver with Master: f90ac71f14084b49aa9a88ecf88c9dee (127.0.191.195:43017)
I20260926 06:56:37.881748 19915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.191.195@KRBTEST.COM'} at 127.0.191.195:36357
I20260926 06:56:37.882023 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:56:37.889272 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.883409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43574 (local address 127.0.191.254:41329)
0926 06:56:37.883472 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:37.883474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:37.883572 (+ 98us) server_negotiation.cc:407] Connection header received
0926 06:56:37.883633 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:37.883634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:37.883642 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:37.883689 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:37.884175 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.884685 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.885452 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.885711 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.885777 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.886996 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:37.886998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:37.886998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:37.887004 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:37.888272 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:37.888604 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:37.888605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:37.888605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:37.888663 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:37.888832 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:37.888832 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:37.888833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:37.888976 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:37.889059 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:37.889144 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 06:56:37.889207 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 06:56:37.890815 19915 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:56:37.892364 19915 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:56:37.898981 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.894591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:43017 (local address 127.0.0.1:36230)
0926 06:56:37.894766 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:37.894807 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:37.894875 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:37.895689 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:37.895690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:37.895692 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:37.895727 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:37.895729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.896901 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.896902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:37.897874 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:37.897877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.898721 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.898722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:37.898779 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.898790 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 06:56:37.898852 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 06:56:37.898896 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":16,"thread_start_us":71,"threads_started":1}
I20260926 06:56:37.899016 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.894664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36230 (local address 127.0.191.195:43017)
0926 06:56:37.895440 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:37.895445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:37.895456 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:56:37.895495 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:37.895498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:37.895515 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:37.895597 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:37.895796 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.896804 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.897958 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.898637 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.898716 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.898771 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:37.898885 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 06:56:37.898927 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":81,"threads_started":1}
I20260926 06:56:37.899381 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.894597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:38611 (local address 127.0.0.1:39922)
0926 06:56:37.894819 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:37.894830 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:37.894875 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:37.895238 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:37.895241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:37.895244 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:37.895313 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:37.895315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.896733 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.896734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:37.898085 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:37.898088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.899127 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.899128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:37.899208 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.899212 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:56:37.899272 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 06:56:37.899310 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":39,"thread_start_us":62,"threads_started":1}
I20260926 06:56:37.899422 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.894856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39922 (local address 127.0.191.193:38611)
0926 06:56:37.895050 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:37.895055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:37.895067 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:56:37.895114 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:37.895117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:37.895137 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:37.895226 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:37.895467 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.896615 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.898183 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.898917 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.898986 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.899050 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:37.899305 (+ 255us) server_negotiation.cc:299] Negotiation successful
0926 06:56:37.899346 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":83,"threads_started":1}
I20260926 06:56:37.900128 20040 tablet_service.cc:1467] Processing CreateTablet for tablet 44bbf56871934d61bfacefd617dad4e5 (DEFAULT_TABLE table=test-table [id=1d99e811d8fe4258adb4e7f15d6ba9c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:37.900221 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.894598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:46119 (local address 127.0.0.1:40562)
0926 06:56:37.895100 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:37.895112 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:37.895187 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:37.895387 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:37.895388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:37.895390 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:37.895539 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:37.895542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.896543 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.896544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:37.898085 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:37.898088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.899963 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.899964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:37.900042 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.900045 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:56:37.900115 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 06:56:37.900153 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":72,"spinlock_wait_cycles":768,"thread_start_us":74,"threads_started":1}
I20260926 06:56:37.900239 20387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:37.894856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40562 (local address 127.0.191.194:46119)
0926 06:56:37.895050 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:37.895055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:37.895212 (+ 157us) server_negotiation.cc:407] Connection header received
0926 06:56:37.895258 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:37.895260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:37.895275 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:37.895359 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:37.895635 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.896429 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.899090 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:37.899812 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:37.899875 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:37.899929 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:37.900129 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 06:56:37.900167 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
I20260926 06:56:37.900511 20040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44bbf56871934d61bfacefd617dad4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:37.901989 20308 tablet_service.cc:1467] Processing CreateTablet for tablet 44bbf56871934d61bfacefd617dad4e5 (DEFAULT_TABLE table=test-table [id=1d99e811d8fe4258adb4e7f15d6ba9c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:37.901991 20174 tablet_service.cc:1467] Processing CreateTablet for tablet 44bbf56871934d61bfacefd617dad4e5 (DEFAULT_TABLE table=test-table [id=1d99e811d8fe4258adb4e7f15d6ba9c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:37.902297 20174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44bbf56871934d61bfacefd617dad4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:37.902297 20308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44bbf56871934d61bfacefd617dad4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:37.903615 20388 tablet_bootstrap.cc:492] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e: Bootstrap starting.
I20260926 06:56:37.904287 20388 tablet_bootstrap.cc:654] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:37.904726 20388 log.cc:824] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:37.905472 20388 tablet_bootstrap.cc:492] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e: No bootstrap required, opened a new log
I20260926 06:56:37.905520 20391 tablet_bootstrap.cc:492] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee: Bootstrap starting.
I20260926 06:56:37.905551 20388 ts_tablet_manager.cc:1397] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:56:37.906198 20391 tablet_bootstrap.cc:654] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:37.906834 20391 log.cc:824] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:37.907191 20390 tablet_bootstrap.cc:492] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369: Bootstrap starting.
I20260926 06:56:37.907550 20391 tablet_bootstrap.cc:492] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee: No bootstrap required, opened a new log
I20260926 06:56:37.907634 20391 ts_tablet_manager.cc:1397] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:56:37.907873 20390 tablet_bootstrap.cc:654] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:37.908005 20388 raft_consensus.cc:348] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3fec0560d148ebaf9370d499a64b4e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 38611 } } peers { permanent_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43017 } } peers { permanent_uuid: "1f20f72c2de34d639f0440bd47351369" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46119 } }
I20260926 06:56:37.908236 20388 raft_consensus.cc:374] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:37.908290 20388 raft_consensus.cc:729] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3fec0560d148ebaf9370d499a64b4e, State: Initialized, Role: FOLLOWER
I20260926 06:56:37.908440 20388 consensus_queue.cc:260] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3fec0560d148ebaf9370d499a64b4e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 38611 } } peers { permanent_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43017 } } peers { permanent_uuid: "1f20f72c2de34d639f0440bd47351369" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46119 } }
I20260926 06:56:37.908514 20390 log.cc:824] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:37.908756 20388 ts_tablet_manager.cc:1428] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:56:37.908840 20106 heartbeater.cc:503] Master 127.0.191.254:41329 was elected leader, sending a full tablet report...
I20260926 06:56:37.909350 20390 tablet_bootstrap.cc:492] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369: No bootstrap required, opened a new log
I20260926 06:56:37.909482 20390 ts_tablet_manager.cc:1397] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:56:37.909518 20391 raft_consensus.cc:348] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3fec0560d148ebaf9370d499a64b4e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 38611 } } peers { permanent_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43017 } } peers { permanent_uuid: "1f20f72c2de34d639f0440bd47351369" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46119 } }
I20260926 06:56:37.909682 20391 raft_consensus.cc:374] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:37.909725 20391 raft_consensus.cc:729] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f90ac71f14084b49aa9a88ecf88c9dee, State: Initialized, Role: FOLLOWER
I20260926 06:56:37.909837 20391 consensus_queue.cc:260] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3fec0560d148ebaf9370d499a64b4e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 38611 } } peers { permanent_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43017 } } peers { permanent_uuid: "1f20f72c2de34d639f0440bd47351369" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46119 } }
I20260926 06:56:37.910111 20391 ts_tablet_manager.cc:1428] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 06:56:37.910225 20374 heartbeater.cc:503] Master 127.0.191.254:41329 was elected leader, sending a full tablet report...
I20260926 06:56:37.911573 20390 raft_consensus.cc:348] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3fec0560d148ebaf9370d499a64b4e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 38611 } } peers { permanent_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43017 } } peers { permanent_uuid: "1f20f72c2de34d639f0440bd47351369" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46119 } }
I20260926 06:56:37.911732 20390 raft_consensus.cc:374] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:37.911782 20390 raft_consensus.cc:729] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f20f72c2de34d639f0440bd47351369, State: Initialized, Role: FOLLOWER
I20260926 06:56:37.911903 20390 consensus_queue.cc:260] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3fec0560d148ebaf9370d499a64b4e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 38611 } } peers { permanent_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43017 } } peers { permanent_uuid: "1f20f72c2de34d639f0440bd47351369" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46119 } }
I20260926 06:56:37.912186 20390 ts_tablet_manager.cc:1428] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:56:37.912223 20240 heartbeater.cc:503] Master 127.0.191.254:41329 was elected leader, sending a full tablet report...
W20260926 06:56:37.978360 20241 tablet.cc:2369] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:56:38.052824 20107 tablet.cc:2369] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:56:38.104049 20395 raft_consensus.cc:482] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:56:38.104182 20395 raft_consensus.cc:504] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3fec0560d148ebaf9370d499a64b4e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 38611 } } peers { permanent_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43017 } } peers { permanent_uuid: "1f20f72c2de34d639f0440bd47351369" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46119 } }
I20260926 06:56:38.104552 20395 leader_election.cc:288] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee3fec0560d148ebaf9370d499a64b4e (127.0.191.193:38611), 1f20f72c2de34d639f0440bd47351369 (127.0.191.194:46119)
I20260926 06:56:38.107534 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.104800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:60397 (local address 127.0.191.193:38611)
0926 06:56:38.104851 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.104853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.104872 (+ 19us) server_negotiation.cc:407] Connection header received
0926 06:56:38.104932 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.104933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.104943 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.105004 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:38.105223 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.105828 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.106791 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.107161 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.107218 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.107225 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.107418 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.107462 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:56:38.107533 20376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.104728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:38611 (local address 127.0.191.195:60397)
0926 06:56:38.104808 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:38.104820 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:38.104864 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:38.105050 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:38.105051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:38.105053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:38.105133 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.105135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.105933 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.105933 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.106695 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.106696 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.107248 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.107250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.107338 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.107343 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:56:38.107394 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 06:56:38.107434 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":30}
I20260926 06:56:38.107617 20387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.104856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:43051 (local address 127.0.191.194:46119)
0926 06:56:38.104919 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.104920 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.105017 (+ 97us) server_negotiation.cc:407] Connection header received
0926 06:56:38.105053 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.105054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.105063 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.105127 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:38.105289 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.105767 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.106832 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.107242 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.107314 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.107322 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.107504 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.107546 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 06:56:38.107637 20397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.104782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:46119 (local address 127.0.191.195:43051)
0926 06:56:38.104945 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:38.104955 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:38.105008 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:38.105135 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:38.105135 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:38.105137 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:38.105222 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.105223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.105842 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.105843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.106704 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.106706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.107321 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.107322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.107414 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.107418 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:56:38.107475 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 06:56:38.107513 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":99,"threads_started":1}
I20260926 06:56:38.107954 20194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44bbf56871934d61bfacefd617dad4e5" candidate_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f20f72c2de34d639f0440bd47351369" is_pre_election: true
I20260926 06:56:38.108055 20060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44bbf56871934d61bfacefd617dad4e5" candidate_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee3fec0560d148ebaf9370d499a64b4e" is_pre_election: true
I20260926 06:56:38.108122 20194 raft_consensus.cc:2454] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f90ac71f14084b49aa9a88ecf88c9dee in term 0.
I20260926 06:56:38.108175 20060 raft_consensus.cc:2454] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f90ac71f14084b49aa9a88ecf88c9dee in term 0.
I20260926 06:56:38.108331 20264 leader_election.cc:302] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f20f72c2de34d639f0440bd47351369, f90ac71f14084b49aa9a88ecf88c9dee; no voters:
I20260926 06:56:38.108477 20395 raft_consensus.cc:2781] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:56:38.108559 20395 raft_consensus.cc:482] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:56:38.108589 20395 raft_consensus.cc:3037] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:38.109257 20395 raft_consensus.cc:504] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3fec0560d148ebaf9370d499a64b4e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 38611 } } peers { permanent_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43017 } } peers { permanent_uuid: "1f20f72c2de34d639f0440bd47351369" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46119 } }
I20260926 06:56:38.109427 20395 leader_election.cc:288] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [CANDIDATE]: Term 1 election: Requested vote from peers ee3fec0560d148ebaf9370d499a64b4e (127.0.191.193:38611), 1f20f72c2de34d639f0440bd47351369 (127.0.191.194:46119)
I20260926 06:56:38.109588 20060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44bbf56871934d61bfacefd617dad4e5" candidate_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee3fec0560d148ebaf9370d499a64b4e"
I20260926 06:56:38.109599 20194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44bbf56871934d61bfacefd617dad4e5" candidate_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f20f72c2de34d639f0440bd47351369"
I20260926 06:56:38.109673 20194 raft_consensus.cc:3037] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:38.109679 20060 raft_consensus.cc:3037] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:38.110358 20194 raft_consensus.cc:2454] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f90ac71f14084b49aa9a88ecf88c9dee in term 1.
I20260926 06:56:38.110455 20060 raft_consensus.cc:2454] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f90ac71f14084b49aa9a88ecf88c9dee in term 1.
I20260926 06:56:38.110519 20264 leader_election.cc:302] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f20f72c2de34d639f0440bd47351369, f90ac71f14084b49aa9a88ecf88c9dee; no voters:
I20260926 06:56:38.110642 20395 raft_consensus.cc:2781] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:38.110791 20395 raft_consensus.cc:686] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [term 1 LEADER]: Becoming Leader. State: Replica: f90ac71f14084b49aa9a88ecf88c9dee, State: Running, Role: LEADER
I20260926 06:56:38.110886 20395 consensus_queue.cc:237] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3fec0560d148ebaf9370d499a64b4e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 38611 } } peers { permanent_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43017 } } peers { permanent_uuid: "1f20f72c2de34d639f0440bd47351369" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46119 } }
I20260926 06:56:38.111706 19915 catalog_manager.cc:5495] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee reported cstate change: term changed from 0 to 1, leader changed from <none> to f90ac71f14084b49aa9a88ecf88c9dee (127.0.191.195). New cstate: current_term: 1 leader_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3fec0560d148ebaf9370d499a64b4e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 38611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f90ac71f14084b49aa9a88ecf88c9dee" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 43017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f20f72c2de34d639f0440bd47351369" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46119 } health_report { overall_health: UNKNOWN } } }
W20260926 06:56:38.121102 20375 tablet.cc:2369] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:56:38.124542 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.122389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36236 (local address 127.0.191.195:43017)
0926 06:56:38.122444 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.122446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.122453 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:56:38.122480 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.122481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.122497 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.122553 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:56:38.122709 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.123180 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.123915 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.124093 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.124143 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.124268 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:56:38.124365 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:56:38.124425 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.124456 (+ 31us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.124471 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:56:38.126359 20194 raft_consensus.cc:1261] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369 [term 1 FOLLOWER]: Refusing update from remote peer f90ac71f14084b49aa9a88ecf88c9dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:56:38.126449 20060 raft_consensus.cc:1261] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e [term 1 FOLLOWER]: Refusing update from remote peer f90ac71f14084b49aa9a88ecf88c9dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:56:38.126714 20395 consensus_queue.cc:1035] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee3fec0560d148ebaf9370d499a64b4e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 38611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:56:38.126888 20398 consensus_queue.cc:1035] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f20f72c2de34d639f0440bd47351369" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 46119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:56:38.129673 20402 mvcc.cc:205] Tried to move back new op lower bound from 7333502149122777088 to 7333502149062787072. Current Snapshot: MvccSnapshot[applied={T|T < 7333502149122777088}]
I20260926 06:56:38.129843 20401 mvcc.cc:205] Tried to move back new op lower bound from 7333502149122777088 to 7333502149062787072. Current Snapshot: MvccSnapshot[applied={T|T < 7333502149122777088 or (T in {7333502149122777088})}]
I20260926 06:56:38.131393 19914 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:56:38.131493 19914 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:56:38.132236 19914 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01939f4def7c4f9b907fab587784e2cb: Sending DeleteTablet for 3 replicas of tablet 44bbf56871934d61bfacefd617dad4e5
I20260926 06:56:38.132556 20308 tablet_service.cc:1514] Processing DeleteTablet for tablet 44bbf56871934d61bfacefd617dad4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:38 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:36230
I20260926 06:56:38.132619 20040 tablet_service.cc:1514] Processing DeleteTablet for tablet 44bbf56871934d61bfacefd617dad4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:38 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:39922
I20260926 06:56:38.132800 20411 tablet_replica.cc:330] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee: stopping tablet replica
I20260926 06:56:38.132812 20174 tablet_service.cc:1514] Processing DeleteTablet for tablet 44bbf56871934d61bfacefd617dad4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:38 UTC) from {username='kudu', principal='kudu/127.0.191.254@KRBTEST.COM'} at 127.0.0.1:40562
I20260926 06:56:38.132892 20411 raft_consensus.cc:2229] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:56:38.132989 20411 raft_consensus.cc:2258] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:56:38.133019 20412 tablet_replica.cc:330] T 44bbf56871934d61bfacefd617dad4e5 P ee3fec0560d148ebaf9370d499a64b4e: stopping tablet replica
I20260926 06:56:38.133225 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 19977
I20260926 06:56:38.133291 20413 tablet_replica.cc:330] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369: stopping tablet replica
I20260926 06:56:38.134069 20413 raft_consensus.cc:2229] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:56:38.134167 20413 raft_consensus.cc:2258] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:56:38.134517 20411 ts_tablet_manager.cc:1905] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:56:38.134687 20413 ts_tablet_manager.cc:1905] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:56:38.136363 20411 ts_tablet_manager.cc:1918] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:56:38.136438 20411 log.cc:1196] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/44bbf56871934d61bfacefd617dad4e5
I20260926 06:56:38.136539 20413 ts_tablet_manager.cc:1918] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:56:38.136601 20413 log.cc:1196] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/44bbf56871934d61bfacefd617dad4e5
I20260926 06:56:38.136722 20411 ts_tablet_manager.cc:1939] T 44bbf56871934d61bfacefd617dad4e5 P f90ac71f14084b49aa9a88ecf88c9dee: Deleting consensus metadata
I20260926 06:56:38.136874 20413 ts_tablet_manager.cc:1939] T 44bbf56871934d61bfacefd617dad4e5 P 1f20f72c2de34d639f0440bd47351369: Deleting consensus metadata
I20260926 06:56:38.137102 19899 catalog_manager.cc:4841] TS f90ac71f14084b49aa9a88ecf88c9dee (127.0.191.195:43017): tablet 44bbf56871934d61bfacefd617dad4e5 (table test-table [id=1d99e811d8fe4258adb4e7f15d6ba9c5]) successfully deleted
I20260926 06:56:38.137420 19902 catalog_manager.cc:4841] TS 1f20f72c2de34d639f0440bd47351369 (127.0.191.194:46119): tablet 44bbf56871934d61bfacefd617dad4e5 (table test-table [id=1d99e811d8fe4258adb4e7f15d6ba9c5]) successfully deleted
W20260926 06:56:38.138686 19901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:38611 (error 108)
I20260926 06:56:38.139180 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.139042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:38611 (local address 127.0.0.1:39928)
0926 06:56:38.139103 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:38.139146 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:38611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 06:56:38.139187 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 20111
W20260926 06:56:38.140537 19901 catalog_manager.cc:4584] TS ee3fec0560d148ebaf9370d499a64b4e (127.0.191.193:38611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 44bbf56871934d61bfacefd617dad4e5: Network error: Client connection negotiation failed: client connection to 127.0.191.193:38611: connect: Connection refused (error 111)
I20260926 06:56:38.145758 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 20245
I20260926 06:56:38.149953 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 19883
2026-09-26T06:56:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2096 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T06:56:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:38Z Disabled control of system clock
I20260926 06:56:38.177201 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:32917
--webserver_interface=127.0.191.254
--webserver_port=0
--builtin_ntp_servers=127.0.191.212:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:32917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:56:38.254323 20418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:38.254532 20418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:38.254552 20418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:38.255817 20418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:38.255865 20418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:38.255878 20418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:38.255892 20418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:38.255904 20418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:38.257179 20418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:32917
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:32917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:38.257364 20418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:38.257695 20418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:38.262467 20426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:38.262480 20424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:38.262472 20423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:38.262502 20418 server_base.cc:1034] running on GCE node
I20260926 06:56:38.262949 20418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:38.263247 20418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:38.264420 20418 hybrid_clock.cc:648] HybridClock initialized: now 1790405798264407 us; error 44 us; skew 500 ppm
I20260926 06:56:38.265766 20418 webserver.cc:460] Webserver started at http://127.0.191.254:44189/ using document root <none> and password file <none>
I20260926 06:56:38.266026 20418 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:38.266099 20418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:38.266248 20418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:38.267314 20418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "65cb0ab932304905bd1aa58ed82eb2d2"
format_stamp: "Formatted at 2026-09-26 06:56:38 on dist-test-slave-ws74"
server_key: "a6c259977374046860758e40de8ad2fd"
server_key_iv: "071771eb6aea3c3be8b8268f328ba1d5"
server_key_version: "encryptionkey@0"
I20260926 06:56:38.267700 20418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "65cb0ab932304905bd1aa58ed82eb2d2"
format_stamp: "Formatted at 2026-09-26 06:56:38 on dist-test-slave-ws74"
server_key: "a6c259977374046860758e40de8ad2fd"
server_key_iv: "071771eb6aea3c3be8b8268f328ba1d5"
server_key_version: "encryptionkey@0"
I20260926 06:56:38.269002 20418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:56:38.269773 20432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:38.269941 20418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:38.270049 20418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "65cb0ab932304905bd1aa58ed82eb2d2"
format_stamp: "Formatted at 2026-09-26 06:56:38 on dist-test-slave-ws74"
server_key: "a6c259977374046860758e40de8ad2fd"
server_key_iv: "071771eb6aea3c3be8b8268f328ba1d5"
server_key_version: "encryptionkey@0"
I20260926 06:56:38.270123 20418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:38.289192 20418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:38.289561 20418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:38.289714 20418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:38.293349 20418 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:32917
I20260926 06:56:38.293424 20484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:32917 every 8 connection(s)
I20260926 06:56:38.293764 20418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:38.294732 20485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:38.296967 20485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2: Bootstrap starting.
I20260926 06:56:38.297574 20485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:38.297878 20485 log.cc:824] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:38.298622 20485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2: No bootstrap required, opened a new log
I20260926 06:56:38.300232 20485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65cb0ab932304905bd1aa58ed82eb2d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 32917 } }
I20260926 06:56:38.300376 20485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:38.300441 20485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65cb0ab932304905bd1aa58ed82eb2d2, State: Initialized, Role: FOLLOWER
I20260926 06:56:38.300598 20485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65cb0ab932304905bd1aa58ed82eb2d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 32917 } }
I20260926 06:56:38.300661 20485 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:38.300683 20485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:38.300743 20485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:38.301229 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 20418
I20260926 06:56:38.301254 20485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65cb0ab932304905bd1aa58ed82eb2d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 32917 } }
I20260926 06:56:38.301352 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:38.301357 20485 leader_election.cc:302] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65cb0ab932304905bd1aa58ed82eb2d2; no voters:
I20260926 06:56:38.301568 20485 leader_election.cc:288] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:38.301586 767 external_mini_cluster.cc:1428] Setting key 8ce873bd595e2e424a5fa46af4a0f8d7
I20260926 06:56:38.301600 20488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:38.301724 20488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [term 1 LEADER]: Becoming Leader. State: Replica: 65cb0ab932304905bd1aa58ed82eb2d2, State: Running, Role: LEADER
I20260926 06:56:38.301831 20485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:38.301815 20488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65cb0ab932304905bd1aa58ed82eb2d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 32917 } }
I20260926 06:56:38.302140 20488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65cb0ab932304905bd1aa58ed82eb2d2. Latest consensus state: current_term: 1 leader_uuid: "65cb0ab932304905bd1aa58ed82eb2d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65cb0ab932304905bd1aa58ed82eb2d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 32917 } } }
I20260926 06:56:38.302219 20488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:38.302354 20489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65cb0ab932304905bd1aa58ed82eb2d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65cb0ab932304905bd1aa58ed82eb2d2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 32917 } } }
I20260926 06:56:38.302493 20489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2 [sys.catalog]: This master's current role is: LEADER
W20260926 06:56:38.303653 20505 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:56:38.303722 20505 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:56:38.303790 20496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:38.303954 20496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:38.305449 20496 catalog_manager.cc:1347] Generated new cluster ID: 92572d5766c94d138eeee827176609d7
I20260926 06:56:38.305511 20496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:38.311596 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.302137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.0.191.254:32917)
0926 06:56:38.302353 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.302363 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.302613 (+ 250us) server_negotiation.cc:407] Connection header received
0926 06:56:38.302853 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.302861 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.302957 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.303403 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:38.303499 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.304862 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.305925 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.306554 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.306688 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.310887 (+ 4199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:38.310891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:38.310891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:38.311107 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:38.311149 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:38.311164 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:38.311239 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.311307 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.311390 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260926 06:56:38.324505 20496 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:38.325139 20496 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:38.334417 20496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2: Generated new TSK 0
I20260926 06:56:38.334618 20496 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:38.337771 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.193:0
--local_ip_for_outbound_sockets=127.0.191.193
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:32917
--builtin_ntp_servers=127.0.191.212:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:56:38.423514 20509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:38.423746 20509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:38.423782 20509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:38.424932 20509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:38.424994 20509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:38.425035 20509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.193
I20260926 06:56:38.426448 20509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.191.193
--webserver_port=0
--tserver_master_addrs=127.0.191.254:32917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.193
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:38.426720 20509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:38.427014 20509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:38.431985 20517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:38.432098 20514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:38.431985 20515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:38.432615 20509 server_base.cc:1034] running on GCE node
I20260926 06:56:38.432864 20509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:38.433115 20509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:38.434280 20509 hybrid_clock.cc:648] HybridClock initialized: now 1790405798434268 us; error 31 us; skew 500 ppm
I20260926 06:56:38.435482 20509 webserver.cc:460] Webserver started at http://127.0.191.193:38651/ using document root <none> and password file <none>
I20260926 06:56:38.435741 20509 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:38.435810 20509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:38.435953 20509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:38.436969 20509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/data/instance:
uuid: "fb863d60c36f4eaf8a8b65155290164e"
format_stamp: "Formatted at 2026-09-26 06:56:38 on dist-test-slave-ws74"
server_key: "8c585d93e4d716007717fee2b372d915"
server_key_iv: "694a27bbf5fa34dc1a6c02007053cd23"
server_key_version: "encryptionkey@0"
I20260926 06:56:38.437352 20509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/wal/instance:
uuid: "fb863d60c36f4eaf8a8b65155290164e"
format_stamp: "Formatted at 2026-09-26 06:56:38 on dist-test-slave-ws74"
server_key: "8c585d93e4d716007717fee2b372d915"
server_key_iv: "694a27bbf5fa34dc1a6c02007053cd23"
server_key_version: "encryptionkey@0"
I20260926 06:56:38.438699 20509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:38.439494 20523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:38.439670 20509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:38.439765 20509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/wal
uuid: "fb863d60c36f4eaf8a8b65155290164e"
format_stamp: "Formatted at 2026-09-26 06:56:38 on dist-test-slave-ws74"
server_key: "8c585d93e4d716007717fee2b372d915"
server_key_iv: "694a27bbf5fa34dc1a6c02007053cd23"
server_key_version: "encryptionkey@0"
I20260926 06:56:38.439850 20509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:38.448247 20509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:38.448520 20509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:38.448665 20509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:38.448871 20509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:38.449182 20509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:38.449237 20509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:38.449277 20509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:38.449306 20509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:38.455461 20509 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.193:42917
I20260926 06:56:38.455610 20636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.193:42917 every 8 connection(s)
I20260926 06:56:38.455871 20509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/data/info.pb
I20260926 06:56:38.461683 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.456616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.193:50541 (local address 127.0.191.254:32917)
0926 06:56:38.456665 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.456667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.456926 (+ 259us) server_negotiation.cc:407] Connection header received
0926 06:56:38.457557 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.457558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.457569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.457623 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:38.458513 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.459018 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.460289 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.460466 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.460527 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.461411 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:38.461413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:38.461413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:38.461419 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:38.461434 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:38.461439 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:38.461489 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.461603 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.461626 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:56:38.461916 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.456601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:32917 (local address 127.0.191.193:50541)
0926 06:56:38.456784 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:38.456824 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:38.457462 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:38.457691 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:38.457695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:38.457705 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:38.458423 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.458429 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.459104 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.459105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.460207 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.460209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.460546 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.460547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.460686 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.461218 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:56:38.461218 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:38.461255 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:56:38.461283 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:38.461514 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:38.461520 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:38.461523 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:56:38.461598 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:56:38.461700 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260926 06:56:38.462445 20637 heartbeater.cc:347] Connected to a master server at 127.0.191.254:32917
I20260926 06:56:38.462525 20637 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:38.462656 20637 heartbeater.cc:511] Master 127.0.191.254:32917 requested a full tablet report, sending...
I20260926 06:56:38.463033 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 20509
I20260926 06:56:38.463145 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-0/wal/instance
I20260926 06:56:38.463106 20449 ts_manager.cc:194] Registered new tserver with Master: fb863d60c36f4eaf8a8b65155290164e (127.0.191.193:42917)
I20260926 06:56:38.463339 767 external_mini_cluster.cc:1428] Setting key a67277b9cefd3c2a5d3dd4c89958f33f
I20260926 06:56:38.463881 20449 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.191.193:50541
I20260926 06:56:38.464378 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.194:0
--local_ip_for_outbound_sockets=127.0.191.194
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:32917
--builtin_ntp_servers=127.0.191.212:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:56:38.543371 20640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:38.543628 20640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:38.543651 20640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:38.544797 20640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:38.544835 20640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:38.544862 20640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.194
I20260926 06:56:38.546221 20640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.191.194
--webserver_port=0
--tserver_master_addrs=127.0.191.254:32917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.194
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:38.546464 20640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:38.546751 20640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:38.551652 20648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:38.551649 20645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:38.551755 20646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:38.551868 20640 server_base.cc:1034] running on GCE node
I20260926 06:56:38.552083 20640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:38.552331 20640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:38.553431 20640 hybrid_clock.cc:648] HybridClock initialized: now 1790405798553399 us; error 45 us; skew 500 ppm
I20260926 06:56:38.554751 20640 webserver.cc:460] Webserver started at http://127.0.191.194:40821/ using document root <none> and password file <none>
I20260926 06:56:38.554989 20640 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:38.555038 20640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:38.555190 20640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:38.556348 20640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/data/instance:
uuid: "f859804f64d840c9882841eb31b2dfba"
format_stamp: "Formatted at 2026-09-26 06:56:38 on dist-test-slave-ws74"
server_key: "d6825a50d5fe5baf0eeb1da7f06f01a5"
server_key_iv: "dc2432dd16dbb474ad5b0093945347b0"
server_key_version: "encryptionkey@0"
I20260926 06:56:38.556707 20640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/wal/instance:
uuid: "f859804f64d840c9882841eb31b2dfba"
format_stamp: "Formatted at 2026-09-26 06:56:38 on dist-test-slave-ws74"
server_key: "d6825a50d5fe5baf0eeb1da7f06f01a5"
server_key_iv: "dc2432dd16dbb474ad5b0093945347b0"
server_key_version: "encryptionkey@0"
I20260926 06:56:38.558038 20640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:56:38.558761 20654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:38.558935 20640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:56:38.559023 20640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/wal
uuid: "f859804f64d840c9882841eb31b2dfba"
format_stamp: "Formatted at 2026-09-26 06:56:38 on dist-test-slave-ws74"
server_key: "d6825a50d5fe5baf0eeb1da7f06f01a5"
server_key_iv: "dc2432dd16dbb474ad5b0093945347b0"
server_key_version: "encryptionkey@0"
I20260926 06:56:38.559077 20640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:38.575605 20640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:38.575878 20640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:38.575976 20640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:38.576157 20640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:38.576421 20640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:38.576450 20640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:38.576471 20640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:38.576484 20640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:38.582412 20640 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.194:33911
I20260926 06:56:38.582482 20767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.194:33911 every 8 connection(s)
I20260926 06:56:38.582811 20640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/data/info.pb
I20260926 06:56:38.588095 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.583584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.194:54891 (local address 127.0.191.254:32917)
0926 06:56:38.583648 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.583650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.583926 (+ 276us) server_negotiation.cc:407] Connection header received
0926 06:56:38.584463 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.584464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.584477 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.584541 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:38.585210 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.585783 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.586749 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.586952 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.587039 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.587795 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:38.587797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:38.587797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:38.587801 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:38.587816 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:38.587820 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:38.587872 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.587989 (+ 117us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.588018 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:56:38.588299 20770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.583582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:32917 (local address 127.0.191.194:54891)
0926 06:56:38.583791 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:38.583824 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:38.584341 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:38.584616 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:38.584620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:38.584627 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:38.585115 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.585119 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.585883 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.585884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.586659 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.586661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.587049 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.587050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.587176 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.587659 (+ 483us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:56:38.587659 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:38.587687 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:56:38.587708 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:38.587904 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:38.587908 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:38.587910 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:56:38.587979 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 06:56:38.588075 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
I20260926 06:56:38.588801 20768 heartbeater.cc:347] Connected to a master server at 127.0.191.254:32917
I20260926 06:56:38.588918 20768 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:38.589083 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 20640
I20260926 06:56:38.589135 20768 heartbeater.cc:511] Master 127.0.191.254:32917 requested a full tablet report, sending...
I20260926 06:56:38.589182 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/wal/instance
I20260926 06:56:38.589437 767 external_mini_cluster.cc:1428] Setting key fca8707affd4718524c1378dda452b8f
I20260926 06:56:38.589543 20449 ts_manager.cc:194] Registered new tserver with Master: f859804f64d840c9882841eb31b2dfba (127.0.191.194:33911)
I20260926 06:56:38.589944 20449 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.191.194:54891
I20260926 06:56:38.590406 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.191.195:0
--local_ip_for_outbound_sockets=127.0.191.195
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:32917
--builtin_ntp_servers=127.0.191.212:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:56:38.673629 20771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:38.673857 20771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:38.673879 20771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:38.674952 20771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:38.674985 20771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:38.675011 20771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.191.195
I20260926 06:56:38.676214 20771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.191.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.191.195
--webserver_port=0
--tserver_master_addrs=127.0.191.254:32917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.191.195
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:38.676380 20771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:38.676609 20771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:38.681522 20777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:38.681591 20771 server_base.cc:1034] running on GCE node
W20260926 06:56:38.681559 20776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:38.681536 20779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:38.681984 20771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:38.682230 20771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:38.683393 20771 hybrid_clock.cc:648] HybridClock initialized: now 1790405798683375 us; error 35 us; skew 500 ppm
I20260926 06:56:38.684715 20771 webserver.cc:460] Webserver started at http://127.0.191.195:46471/ using document root <none> and password file <none>
I20260926 06:56:38.684969 20771 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:38.685036 20771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:38.685168 20771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:38.686237 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/data/instance:
uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279"
format_stamp: "Formatted at 2026-09-26 06:56:38 on dist-test-slave-ws74"
server_key: "5ed4a9b5224d3761bc747b3fad2333e9"
server_key_iv: "48c19feb7fe0ad1430dea4ece7f7587d"
server_key_version: "encryptionkey@0"
I20260926 06:56:38.686592 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/wal/instance:
uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279"
format_stamp: "Formatted at 2026-09-26 06:56:38 on dist-test-slave-ws74"
server_key: "5ed4a9b5224d3761bc747b3fad2333e9"
server_key_iv: "48c19feb7fe0ad1430dea4ece7f7587d"
server_key_version: "encryptionkey@0"
I20260926 06:56:38.687808 20771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:56:38.688548 20785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:38.688751 20771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:38.688848 20771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/wal
uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279"
format_stamp: "Formatted at 2026-09-26 06:56:38 on dist-test-slave-ws74"
server_key: "5ed4a9b5224d3761bc747b3fad2333e9"
server_key_iv: "48c19feb7fe0ad1430dea4ece7f7587d"
server_key_version: "encryptionkey@0"
I20260926 06:56:38.688931 20771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:38.702338 20771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:38.702629 20771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:38.702785 20771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:38.703008 20771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:56:38.703382 20771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:56:38.703440 20771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:38.703462 20771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:56:38.703512 20771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:38.709064 20771 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.195:37067
I20260926 06:56:38.709110 20898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.195:37067 every 8 connection(s)
I20260926 06:56:38.709429 20771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/data/info.pb
I20260926 06:56:38.714098 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.710078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:60039 (local address 127.0.191.254:32917)
0926 06:56:38.710123 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.710125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.710391 (+ 266us) server_negotiation.cc:407] Connection header received
0926 06:56:38.710837 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.710837 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.710847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.710907 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:38.711532 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.712009 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.712914 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.713089 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.713139 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.713873 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:38.713874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:38.713875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:38.713879 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:38.713891 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:38.713895 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:38.713938 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.714020 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.714042 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:56:38.714273 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.710059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:32917 (local address 127.0.191.195:60039)
0926 06:56:38.710238 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:38.710277 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:38.710756 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:38.710979 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:38.710983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:38.710990 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:56:38.711463 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.711467 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.712079 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.712080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.712834 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.712835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.713158 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.713159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.713261 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.713747 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:56:38.713748 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:56:38.713774 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:56:38.713795 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:56:38.713949 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:56:38.713953 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:56:38.713955 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:56:38.714007 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 06:56:38.714080 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260926 06:56:38.714735 20899 heartbeater.cc:347] Connected to a master server at 127.0.191.254:32917
I20260926 06:56:38.714834 20899 heartbeater.cc:464] Registering TS with master...
I20260926 06:56:38.715021 20899 heartbeater.cc:511] Master 127.0.191.254:32917 requested a full tablet report, sending...
I20260926 06:56:38.715233 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 20771
I20260926 06:56:38.715348 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/wal/instance
I20260926 06:56:38.715446 20449 ts_manager.cc:194] Registered new tserver with Master: cedcf5b2d9c646a7a7b2c5d0fa4f9279 (127.0.191.195:37067)
I20260926 06:56:38.715555 767 external_mini_cluster.cc:1428] Setting key 74fe839f08671d4b965e5115870919c3
I20260926 06:56:38.715862 20449 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.191.195:60039
I20260926 06:56:38.716483 767 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:56:38.720561 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.717783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53342 (local address 127.0.191.254:32917)
0926 06:56:38.717850 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.717851 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.718003 (+ 152us) server_negotiation.cc:407] Connection header received
0926 06:56:38.718065 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.718066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.718077 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.718126 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:38.718284 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.718738 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.719494 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.719695 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.719771 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.720333 (+ 562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:38.720334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:38.720335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:38.720339 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:38.720357 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:38.720361 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:38.720403 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.720464 (+ 61us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.720484 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 06:56:38.721935 20449 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:56:38.723347 20449 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:56:38.729223 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.725253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:33911 (local address 127.0.0.1:44096)
0926 06:56:38.725423 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:38.725459 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:38.725535 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:38.725960 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:38.725961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:38.725963 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:38.726008 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.726010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.726933 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.726934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.728086 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.728088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.728956 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.728956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.729018 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.729028 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 06:56:38.729089 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 06:56:38.729124 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":51,"threads_started":1}
I20260926 06:56:38.729229 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.725391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44096 (local address 127.0.191.194:33911)
0926 06:56:38.725817 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.725822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.725832 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:56:38.725867 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.725869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.725882 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.725950 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:38.726086 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.726838 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.728177 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.728873 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.728947 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.729029 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.729102 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.729144 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":84,"threads_started":1}
I20260926 06:56:38.729662 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.725483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.195:37067 (local address 127.0.0.1:37178)
0926 06:56:38.725656 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:38.725665 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:38.725705 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:38.725974 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:38.725975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:38.725976 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:38.726012 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.726013 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.727318 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.727319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.728339 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.728341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.729180 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.729180 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.729242 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.729246 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:56:38.729534 (+ 288us) client_negotiation.cc:240] Negotiation successful
0926 06:56:38.729581 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":56,"threads_started":1}
I20260926 06:56:38.729949 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.725645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37178 (local address 127.0.191.195:37067)
0926 06:56:38.725780 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.725784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.725793 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:56:38.725826 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.725828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.725844 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.725922 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:38.726159 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.727223 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.728437 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.729094 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.729162 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.729241 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.729831 (+ 590us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.729871 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260926 06:56:38.730062 20702 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf0828cff754dadb7db3d42792a5e26 (DEFAULT_TABLE table=test-table [id=b7599e1bdb37462894d943ef458a0ae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:38.730386 20702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf0828cff754dadb7db3d42792a5e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:38.730511 20833 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf0828cff754dadb7db3d42792a5e26 (DEFAULT_TABLE table=test-table [id=b7599e1bdb37462894d943ef458a0ae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:38.730856 20833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf0828cff754dadb7db3d42792a5e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:38.731545 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.725276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:42917 (local address 127.0.0.1:34004)
0926 06:56:38.725567 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:38.725577 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:38.725636 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:38.725882 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:38.725885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:38.725887 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:38.725946 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.725948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.729278 (+ 3330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.729279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.730389 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.730392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.731312 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.731314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.731387 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.731390 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:56:38.731446 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 06:56:38.731482 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":86,"threads_started":1}
I20260926 06:56:38.731592 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.725331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34004 (local address 127.0.191.193:42917)
0926 06:56:38.725538 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.725543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.725705 (+ 162us) server_negotiation.cc:407] Connection header received
0926 06:56:38.725752 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.725755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.725771 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.725866 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:38.726119 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.727229 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.730494 (+ 3265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.731202 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.731281 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.731374 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.731471 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.731511 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":107,"threads_started":1}
I20260926 06:56:38.732213 20571 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf0828cff754dadb7db3d42792a5e26 (DEFAULT_TABLE table=test-table [id=b7599e1bdb37462894d943ef458a0ae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:56:38.732520 20571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf0828cff754dadb7db3d42792a5e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:38.733223 20913 tablet_bootstrap.cc:492] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba: Bootstrap starting.
I20260926 06:56:38.733805 20913 tablet_bootstrap.cc:654] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:38.734028 20913 log.cc:824] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:38.734256 20914 tablet_bootstrap.cc:492] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279: Bootstrap starting.
I20260926 06:56:38.734757 20914 tablet_bootstrap.cc:654] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:38.735035 20914 log.cc:824] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:38.735702 20917 tablet_bootstrap.cc:492] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e: Bootstrap starting.
I20260926 06:56:38.735742 20914 tablet_bootstrap.cc:492] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279: No bootstrap required, opened a new log
I20260926 06:56:38.735801 20914 ts_tablet_manager.cc:1397] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:56:38.735911 20913 tablet_bootstrap.cc:492] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba: No bootstrap required, opened a new log
I20260926 06:56:38.735980 20913 ts_tablet_manager.cc:1397] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 06:56:38.736407 20917 tablet_bootstrap.cc:654] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:38.736682 20917 log.cc:824] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:38.737326 20917 tablet_bootstrap.cc:492] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e: No bootstrap required, opened a new log
I20260926 06:56:38.737424 20917 ts_tablet_manager.cc:1397] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:56:38.738395 20914 raft_consensus.cc:348] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb863d60c36f4eaf8a8b65155290164e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42917 } } peers { permanent_uuid: "f859804f64d840c9882841eb31b2dfba" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33911 } } peers { permanent_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37067 } }
I20260926 06:56:38.738440 20913 raft_consensus.cc:348] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb863d60c36f4eaf8a8b65155290164e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42917 } } peers { permanent_uuid: "f859804f64d840c9882841eb31b2dfba" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33911 } } peers { permanent_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37067 } }
I20260926 06:56:38.738562 20914 raft_consensus.cc:374] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:38.738562 20913 raft_consensus.cc:374] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:38.738597 20914 raft_consensus.cc:729] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cedcf5b2d9c646a7a7b2c5d0fa4f9279, State: Initialized, Role: FOLLOWER
I20260926 06:56:38.738597 20913 raft_consensus.cc:729] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f859804f64d840c9882841eb31b2dfba, State: Initialized, Role: FOLLOWER
I20260926 06:56:38.738723 20914 consensus_queue.cc:260] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb863d60c36f4eaf8a8b65155290164e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42917 } } peers { permanent_uuid: "f859804f64d840c9882841eb31b2dfba" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33911 } } peers { permanent_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37067 } }
I20260926 06:56:38.738714 20913 consensus_queue.cc:260] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb863d60c36f4eaf8a8b65155290164e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42917 } } peers { permanent_uuid: "f859804f64d840c9882841eb31b2dfba" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33911 } } peers { permanent_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37067 } }
I20260926 06:56:38.738968 20914 ts_tablet_manager.cc:1428] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:56:38.738968 20913 ts_tablet_manager.cc:1428] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:56:38.739095 20899 heartbeater.cc:503] Master 127.0.191.254:32917 was elected leader, sending a full tablet report...
I20260926 06:56:38.739363 20768 heartbeater.cc:503] Master 127.0.191.254:32917 was elected leader, sending a full tablet report...
I20260926 06:56:38.740051 20917 raft_consensus.cc:348] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb863d60c36f4eaf8a8b65155290164e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42917 } } peers { permanent_uuid: "f859804f64d840c9882841eb31b2dfba" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33911 } } peers { permanent_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37067 } }
I20260926 06:56:38.740211 20917 raft_consensus.cc:374] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:38.740247 20917 raft_consensus.cc:729] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb863d60c36f4eaf8a8b65155290164e, State: Initialized, Role: FOLLOWER
I20260926 06:56:38.740356 20917 consensus_queue.cc:260] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb863d60c36f4eaf8a8b65155290164e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42917 } } peers { permanent_uuid: "f859804f64d840c9882841eb31b2dfba" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33911 } } peers { permanent_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37067 } }
I20260926 06:56:38.740622 20917 ts_tablet_manager.cc:1428] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:56:38.740659 20637 heartbeater.cc:503] Master 127.0.191.254:32917 was elected leader, sending a full tablet report...
I20260926 06:56:38.825193 20920 raft_consensus.cc:482] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:56:38.825316 20920 raft_consensus.cc:504] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb863d60c36f4eaf8a8b65155290164e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42917 } } peers { permanent_uuid: "f859804f64d840c9882841eb31b2dfba" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33911 } } peers { permanent_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37067 } }
I20260926 06:56:38.825727 20920 leader_election.cc:288] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb863d60c36f4eaf8a8b65155290164e (127.0.191.193:42917), f859804f64d840c9882841eb31b2dfba (127.0.191.194:33911)
I20260926 06:56:38.828433 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.825885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:42917 (local address 127.0.191.195:45041)
0926 06:56:38.825934 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:38.825948 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:38.825995 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:38.826136 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:38.826137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:38.826138 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:38.826212 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.826213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.826849 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.826851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.827590 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.827592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.828149 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.828150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.828221 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.828224 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:56:38.828317 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 06:56:38.828360 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 06:56:38.828433 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.825978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:45041 (local address 127.0.191.193:42917)
0926 06:56:38.826026 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.826028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.826040 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:56:38.826067 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.826068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.826077 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.826121 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:38.826284 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.826760 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.827687 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.828067 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.828153 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.828161 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.828326 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.828368 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 06:56:38.828548 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.825895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.194:33911 (local address 127.0.191.195:39995)
0926 06:56:38.826088 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:38.826096 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:56:38.826168 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:56:38.826324 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:56:38.826324 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:56:38.826326 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:56:38.826396 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.826397 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.826994 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.826995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.827730 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:56:38.827732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.828259 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.828260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:56:38.828339 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.828342 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:56:38.828418 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 06:56:38.828454 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":2,"thread_start_us":97,"threads_started":1}
I20260926 06:56:38.828526 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.825973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.191.195:39995 (local address 127.0.191.194:33911)
0926 06:56:38.826030 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.826032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.826155 (+ 123us) server_negotiation.cc:407] Connection header received
0926 06:56:38.826241 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.826242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.826250 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.826305 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:56:38.826474 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.826914 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.827823 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.828177 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.828261 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.828268 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.828427 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.828464 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:56:38.828788 20591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf0828cff754dadb7db3d42792a5e26" candidate_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb863d60c36f4eaf8a8b65155290164e" is_pre_election: true
I20260926 06:56:38.828936 20591 raft_consensus.cc:2454] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cedcf5b2d9c646a7a7b2c5d0fa4f9279 in term 0.
I20260926 06:56:38.828903 20722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf0828cff754dadb7db3d42792a5e26" candidate_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f859804f64d840c9882841eb31b2dfba" is_pre_election: true
I20260926 06:56:38.829079 20722 raft_consensus.cc:2454] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cedcf5b2d9c646a7a7b2c5d0fa4f9279 in term 0.
I20260926 06:56:38.829157 20788 leader_election.cc:302] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cedcf5b2d9c646a7a7b2c5d0fa4f9279, fb863d60c36f4eaf8a8b65155290164e; no voters:
I20260926 06:56:38.829355 20920 raft_consensus.cc:2781] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:56:38.829475 20920 raft_consensus.cc:482] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:56:38.829509 20920 raft_consensus.cc:3037] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:38.830189 20920 raft_consensus.cc:504] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb863d60c36f4eaf8a8b65155290164e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42917 } } peers { permanent_uuid: "f859804f64d840c9882841eb31b2dfba" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33911 } } peers { permanent_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37067 } }
I20260926 06:56:38.830348 20920 leader_election.cc:288] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [CANDIDATE]: Term 1 election: Requested vote from peers fb863d60c36f4eaf8a8b65155290164e (127.0.191.193:42917), f859804f64d840c9882841eb31b2dfba (127.0.191.194:33911)
I20260926 06:56:38.830504 20591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf0828cff754dadb7db3d42792a5e26" candidate_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb863d60c36f4eaf8a8b65155290164e"
I20260926 06:56:38.830505 20722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf0828cff754dadb7db3d42792a5e26" candidate_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f859804f64d840c9882841eb31b2dfba"
I20260926 06:56:38.830577 20591 raft_consensus.cc:3037] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:38.830600 20722 raft_consensus.cc:3037] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:38.831317 20722 raft_consensus.cc:2454] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cedcf5b2d9c646a7a7b2c5d0fa4f9279 in term 1.
I20260926 06:56:38.831333 20591 raft_consensus.cc:2454] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cedcf5b2d9c646a7a7b2c5d0fa4f9279 in term 1.
I20260926 06:56:38.831509 20787 leader_election.cc:302] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cedcf5b2d9c646a7a7b2c5d0fa4f9279, f859804f64d840c9882841eb31b2dfba; no voters:
I20260926 06:56:38.831633 20920 raft_consensus.cc:2781] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:38.831782 20920 raft_consensus.cc:686] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [term 1 LEADER]: Becoming Leader. State: Replica: cedcf5b2d9c646a7a7b2c5d0fa4f9279, State: Running, Role: LEADER
I20260926 06:56:38.831887 20920 consensus_queue.cc:237] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb863d60c36f4eaf8a8b65155290164e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42917 } } peers { permanent_uuid: "f859804f64d840c9882841eb31b2dfba" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33911 } } peers { permanent_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37067 } }
I20260926 06:56:38.832715 20448 catalog_manager.cc:5495] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 reported cstate change: term changed from 0 to 1, leader changed from <none> to cedcf5b2d9c646a7a7b2c5d0fa4f9279 (127.0.191.195). New cstate: current_term: 1 leader_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb863d60c36f4eaf8a8b65155290164e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f859804f64d840c9882841eb31b2dfba" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cedcf5b2d9c646a7a7b2c5d0fa4f9279" member_type: VOTER last_known_addr { host: "127.0.191.195" port: 37067 } health_report { overall_health: HEALTHY } } }
W20260926 06:56:38.833295 20769 tablet.cc:2369] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:56:38.845944 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.843790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37186 (local address 127.0.191.195:37067)
0926 06:56:38.843862 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:38.843864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:38.843872 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:56:38.843901 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:38.843902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:38.843917 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:38.843972 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:56:38.844067 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.844510 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.845219 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:38.845447 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:38.845493 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:38.845628 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:56:38.845718 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:56:38.845778 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:38.845814 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 06:56:38.845828 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 06:56:38.847714 20722 raft_consensus.cc:1261] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba [term 1 FOLLOWER]: Refusing update from remote peer cedcf5b2d9c646a7a7b2c5d0fa4f9279: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:56:38.847713 20591 raft_consensus.cc:1261] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e [term 1 FOLLOWER]: Refusing update from remote peer cedcf5b2d9c646a7a7b2c5d0fa4f9279: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:56:38.848062 20923 consensus_queue.cc:1035] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f859804f64d840c9882841eb31b2dfba" member_type: VOTER last_known_addr { host: "127.0.191.194" port: 33911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:56:38.848165 20920 consensus_queue.cc:1035] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb863d60c36f4eaf8a8b65155290164e" member_type: VOTER last_known_addr { host: "127.0.191.193" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:56:38.850937 20928 mvcc.cc:205] Tried to move back new op lower bound from 7333502152077574144 to 7333502152016150528. Current Snapshot: MvccSnapshot[applied={T|T < 7333502152077574144}]
I20260926 06:56:38.852684 20449 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:56:38.852773 20449 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:56:38.853538 20449 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65cb0ab932304905bd1aa58ed82eb2d2: Sending DeleteTablet for 3 replicas of tablet 9cf0828cff754dadb7db3d42792a5e26
I20260926 06:56:38.853859 20833 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cf0828cff754dadb7db3d42792a5e26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:38 UTC) from {username='slave'} at 127.0.0.1:37178
I20260926 06:56:38.853865 20702 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cf0828cff754dadb7db3d42792a5e26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:38 UTC) from {username='slave'} at 127.0.0.1:44096
I20260926 06:56:38.853853 20571 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cf0828cff754dadb7db3d42792a5e26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:56:38 UTC) from {username='slave'} at 127.0.0.1:34004
I20260926 06:56:38.854113 20935 tablet_replica.cc:330] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e: stopping tablet replica
I20260926 06:56:38.854206 20935 raft_consensus.cc:2229] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:56:38.854306 20935 raft_consensus.cc:2258] T 9cf0828cff754dadb7db3d42792a5e26 P fb863d60c36f4eaf8a8b65155290164e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:56:38.854336 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 20509
I20260926 06:56:38.854461 20936 tablet_replica.cc:330] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba: stopping tablet replica
I20260926 06:56:38.854547 20936 raft_consensus.cc:2229] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:56:38.854631 20936 raft_consensus.cc:2258] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:56:38.854766 20937 tablet_replica.cc:330] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279: stopping tablet replica
I20260926 06:56:38.854846 20937 raft_consensus.cc:2229] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:56:38.854974 20937 raft_consensus.cc:2258] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:56:38.855087 20936 ts_tablet_manager.cc:1905] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:56:38.855393 20937 ts_tablet_manager.cc:1905] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:56:38.857008 20936 ts_tablet_manager.cc:1918] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:56:38.857080 20937 ts_tablet_manager.cc:1918] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:56:38.857156 20937 log.cc:1196] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-2/wal/wals/9cf0828cff754dadb7db3d42792a5e26
I20260926 06:56:38.857174 20936 log.cc:1196] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba: Deleting WAL directory at /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790405559478510-767-0/minicluster-data/ts-1/wal/wals/9cf0828cff754dadb7db3d42792a5e26
I20260926 06:56:38.857453 20937 ts_tablet_manager.cc:1939] T 9cf0828cff754dadb7db3d42792a5e26 P cedcf5b2d9c646a7a7b2c5d0fa4f9279: Deleting consensus metadata
I20260926 06:56:38.857772 20936 ts_tablet_manager.cc:1939] T 9cf0828cff754dadb7db3d42792a5e26 P f859804f64d840c9882841eb31b2dfba: Deleting consensus metadata
I20260926 06:56:38.857831 20435 catalog_manager.cc:4841] TS cedcf5b2d9c646a7a7b2c5d0fa4f9279 (127.0.191.195:37067): tablet 9cf0828cff754dadb7db3d42792a5e26 (table test-table [id=b7599e1bdb37462894d943ef458a0ae8]) successfully deleted
I20260926 06:56:38.858255 20434 catalog_manager.cc:4841] TS f859804f64d840c9882841eb31b2dfba (127.0.191.194:33911): tablet 9cf0828cff754dadb7db3d42792a5e26 (table test-table [id=b7599e1bdb37462894d943ef458a0ae8]) successfully deleted
W20260926 06:56:38.861014 20435 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.191.193:42917 (error 108)
I20260926 06:56:38.861267 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 20640
I20260926 06:56:38.861964 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:38.861476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.193:42917 (local address 127.0.0.1:34018)
0926 06:56:38.861885 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0926 06:56:38.861930 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.193:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381}
W20260926 06:56:38.862258 20435 catalog_manager.cc:4584] TS fb863d60c36f4eaf8a8b65155290164e (127.0.191.193:42917): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9cf0828cff754dadb7db3d42792a5e26: Network error: Client connection negotiation failed: client connection to 127.0.191.193:42917: connect: Connection refused (error 111)
I20260926 06:56:38.866101 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 20771
I20260926 06:56:38.870263 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 20418
2026-09-26T06:56:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (719 ms)
[----------] 6 tests from EncryptionPolicyTest (8366 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:38 dist-test-slave-ws74 krb5kdc[20941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:38 dist-test-slave-ws74 krb5kdc[20941](info): set up 2 sockets
Sep 26 06:56:38 dist-test-slave-ws74 krb5kdc[20941](info): commencing operation
krb5kdc: starting...
W20260926 06:56:39.901425 767 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:39 dist-test-slave-ws74 krb5kdc[20941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:56:40.017189 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41841
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:40.097256 20954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:40.097517 20954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:40.097563 20954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:40.098706 20954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:40.098745 20954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:40.098759 20954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:40.098796 20954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:40.098826 20954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:40.100250 20954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41841
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:40.100461 20954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:40.100665 20954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:40.105461 20959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:40.105546 20962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:40.105619 20954 server_base.cc:1034] running on GCE node
W20260926 06:56:40.105487 20960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:40.105885 20954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:40.106176 20954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:40.107339 20954 hybrid_clock.cc:648] HybridClock initialized: now 1790405800107313 us; error 59 us; skew 500 ppm
Sep 26 06:56:40 dist-test-slave-ws74 krb5kdc[20941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405800, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:40.109859 20954 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:56:40.110329 20954 webserver.cc:460] Webserver started at http://127.0.0.1:42933/ using document root <none> and password file <none>
I20260926 06:56:40.110561 20954 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:40.110630 20954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:40.110770 20954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:40.111735 20954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "0dc1e3bbffe243cf868217be7e613601"
format_stamp: "Formatted at 2026-09-26 06:56:40 on dist-test-slave-ws74"
server_key: "f04d51e4526ec03224a9b1e39cd829f4"
server_key_iv: "9aeda09a2263a1ebc904681b24018041"
server_key_version: "encryptionkey@0"
I20260926 06:56:40.112085 20954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "0dc1e3bbffe243cf868217be7e613601"
format_stamp: "Formatted at 2026-09-26 06:56:40 on dist-test-slave-ws74"
server_key: "f04d51e4526ec03224a9b1e39cd829f4"
server_key_iv: "9aeda09a2263a1ebc904681b24018041"
server_key_version: "encryptionkey@0"
I20260926 06:56:40.113410 20954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:56:40.114122 20969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:40.114295 20954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:40.114389 20954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "0dc1e3bbffe243cf868217be7e613601"
format_stamp: "Formatted at 2026-09-26 06:56:40 on dist-test-slave-ws74"
server_key: "f04d51e4526ec03224a9b1e39cd829f4"
server_key_iv: "9aeda09a2263a1ebc904681b24018041"
server_key_version: "encryptionkey@0"
I20260926 06:56:40.114475 20954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:40.127110 20954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:40.129359 20954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:40.129570 20954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:40.133543 20954 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41841
I20260926 06:56:40.133580 21021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41841 every 8 connection(s)
I20260926 06:56:40.133905 20954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:40.134739 21022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:40.136854 21022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601: Bootstrap starting.
I20260926 06:56:40.137423 21022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:40.137710 21022 log.cc:824] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:40.138363 21022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601: No bootstrap required, opened a new log
I20260926 06:56:40.139878 21022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc1e3bbffe243cf868217be7e613601" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41841 } }
I20260926 06:56:40.140018 21022 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:40.140043 21022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc1e3bbffe243cf868217be7e613601, State: Initialized, Role: FOLLOWER
I20260926 06:56:40.140208 21022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc1e3bbffe243cf868217be7e613601" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41841 } }
I20260926 06:56:40.140269 21022 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:40.140290 21022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:40.140354 21022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:40.140854 21022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc1e3bbffe243cf868217be7e613601" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41841 } }
I20260926 06:56:40.140964 21022 leader_election.cc:302] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dc1e3bbffe243cf868217be7e613601; no voters:
I20260926 06:56:40.141150 21022 leader_election.cc:288] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:40.141209 21025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:40.141417 21022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:40.141405 21025 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [term 1 LEADER]: Becoming Leader. State: Replica: 0dc1e3bbffe243cf868217be7e613601, State: Running, Role: LEADER
I20260926 06:56:40.141566 21025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc1e3bbffe243cf868217be7e613601" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41841 } }
I20260926 06:56:40.141956 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 20954
I20260926 06:56:40.142091 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:40.142316 767 external_mini_cluster.cc:1428] Setting key da677bce7844ea180e839bc9b6f203de
I20260926 06:56:40.142716 21026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dc1e3bbffe243cf868217be7e613601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc1e3bbffe243cf868217be7e613601" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41841 } } }
I20260926 06:56:40.143055 21027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dc1e3bbffe243cf868217be7e613601. Latest consensus state: current_term: 1 leader_uuid: "0dc1e3bbffe243cf868217be7e613601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc1e3bbffe243cf868217be7e613601" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41841 } } }
I20260926 06:56:40.143122 21027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:40.142818 21026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601 [sys.catalog]: This master's current role is: LEADER
W20260926 06:56:40.143431 21039 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:56:40.143493 21039 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:56:40.143572 21042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:40.143787 21042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:40.145529 21042 catalog_manager.cc:1347] Generated new cluster ID: 21ed8cffec964314b38356730166b0ef
I20260926 06:56:40.145578 21042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:56:40 dist-test-slave-ws74 krb5kdc[20941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 06:56:40.154381 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:40.143005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58683 (local address 127.0.0.1:41841)
0926 06:56:40.143164 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:40.143172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:40.143651 (+ 479us) server_negotiation.cc:407] Connection header received
0926 06:56:40.143801 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:40.143809 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:40.143866 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:40.144020 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:40.144811 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:40.146151 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:40.147234 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:40.147822 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:40.147914 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:40.150706 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:40.150710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:40.150710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:40.150722 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:40.152655 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:40.153084 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:40.153087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:40.153087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:40.153208 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:40.153454 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:40.153455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:40.153455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:40.153781 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:40.153944 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:40.154055 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 06:56:40.154161 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260926 06:56:40.155367 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:40.161227 21042 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:40.161768 21042 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:40.167541 21042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dc1e3bbffe243cf868217be7e613601: Generated new TSK 0
I20260926 06:56:40.167728 21042 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:40.172132 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:40.157751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43125 (local address 127.0.0.1:41841)
0926 06:56:40.157787 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:40.157789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:40.160877 (+ 3088us) server_negotiation.cc:407] Connection header received
0926 06:56:40.161013 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:40.161014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:40.161027 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:40.161094 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:40.166031 (+ 4937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:40.166810 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:40.167544 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:40.167873 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:40.167944 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:40.169243 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:40.169244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:40.169244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:40.169252 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:40.170887 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:40.171280 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:40.171281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:40.171281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:40.171336 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:40.171602 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:40.171603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:40.171603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:40.171714 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:40.171822 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:40.171972 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 06:56:40.172035 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:56:40.174309 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 20954
W20260926 06:56:40.178618 20939 connection.cc:535] client connection to 127.0.0.1:41841 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41841): Connection reset by peer (error 104)
2026-09-26T06:56:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1302 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:40 dist-test-slave-ws74 krb5kdc[21053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:40 dist-test-slave-ws74 krb5kdc[21053](info): set up 2 sockets
Sep 26 06:56:40 dist-test-slave-ws74 krb5kdc[21053](info): commencing operation
krb5kdc: starting...
W20260926 06:56:41.221438 767 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:41 dist-test-slave-ws74 krb5kdc[21053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:56:41.341317 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36191
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:41.421052 21066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:41.421283 21066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:41.421320 21066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:41.422472 21066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:41.422541 21066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:41.422572 21066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:41.422603 21066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:41.422627 21066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:41.424090 21066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36191
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:41.424360 21066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:41.424580 21066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:41.429409 21074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:41.429544 21066 server_base.cc:1034] running on GCE node
W20260926 06:56:41.429474 21072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:41.429409 21071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:41.429881 21066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:41.430166 21066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:41.431358 21066 hybrid_clock.cc:648] HybridClock initialized: now 1790405801431337 us; error 37 us; skew 500 ppm
Sep 26 06:56:41 dist-test-slave-ws74 krb5kdc[21053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405801, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:41.433750 21066 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:56:41.434209 21066 webserver.cc:460] Webserver started at http://127.0.0.1:41841/ using document root <none> and password file <none>
I20260926 06:56:41.434453 21066 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:41.434523 21066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:41.434670 21066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:41.435701 21066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "0b6347a292db42dfa1151f58c34403a0"
format_stamp: "Formatted at 2026-09-26 06:56:41 on dist-test-slave-ws74"
server_key: "6a974477ad6f70ee9bda6d9f85b40edb"
server_key_iv: "d0d35d0a31a3cebdaec32bd6ef5bd0b9"
server_key_version: "encryptionkey@0"
I20260926 06:56:41.436053 21066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "0b6347a292db42dfa1151f58c34403a0"
format_stamp: "Formatted at 2026-09-26 06:56:41 on dist-test-slave-ws74"
server_key: "6a974477ad6f70ee9bda6d9f85b40edb"
server_key_iv: "d0d35d0a31a3cebdaec32bd6ef5bd0b9"
server_key_version: "encryptionkey@0"
I20260926 06:56:41.437333 21066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:56:41.438091 21081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:41.438254 21066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:56:41.438359 21066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "0b6347a292db42dfa1151f58c34403a0"
format_stamp: "Formatted at 2026-09-26 06:56:41 on dist-test-slave-ws74"
server_key: "6a974477ad6f70ee9bda6d9f85b40edb"
server_key_iv: "d0d35d0a31a3cebdaec32bd6ef5bd0b9"
server_key_version: "encryptionkey@0"
I20260926 06:56:41.438445 21066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:41.449183 21066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:41.451126 21066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:41.451288 21066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:41.455354 21066 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36191
I20260926 06:56:41.455389 21133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36191 every 8 connection(s)
I20260926 06:56:41.455742 21066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:41.456585 21134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:41.458811 21134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0: Bootstrap starting.
I20260926 06:56:41.459311 21134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:41.459542 21134 log.cc:824] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:41.460156 21134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0: No bootstrap required, opened a new log
I20260926 06:56:41.461740 21134 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6347a292db42dfa1151f58c34403a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36191 } }
I20260926 06:56:41.461874 21134 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:41.461895 21134 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6347a292db42dfa1151f58c34403a0, State: Initialized, Role: FOLLOWER
I20260926 06:56:41.462037 21134 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6347a292db42dfa1151f58c34403a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36191 } }
I20260926 06:56:41.462124 21134 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:41.462148 21134 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:41.462176 21134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:41.462713 21134 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6347a292db42dfa1151f58c34403a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36191 } }
I20260926 06:56:41.462802 21134 leader_election.cc:302] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b6347a292db42dfa1151f58c34403a0; no voters:
I20260926 06:56:41.462917 21134 leader_election.cc:288] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:41.462992 21137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:41.463165 21137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0b6347a292db42dfa1151f58c34403a0, State: Running, Role: LEADER
I20260926 06:56:41.463196 21134 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:41.463254 21137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6347a292db42dfa1151f58c34403a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36191 } }
I20260926 06:56:41.463683 21138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b6347a292db42dfa1151f58c34403a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6347a292db42dfa1151f58c34403a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36191 } } }
I20260926 06:56:41.463718 21139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b6347a292db42dfa1151f58c34403a0. Latest consensus state: current_term: 1 leader_uuid: "0b6347a292db42dfa1151f58c34403a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6347a292db42dfa1151f58c34403a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36191 } } }
I20260926 06:56:41.463872 21139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:41.463840 21138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:41.464273 21142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:41.464811 21142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:41.465837 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 21066
I20260926 06:56:41.465955 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:41.466161 767 external_mini_cluster.cc:1428] Setting key 40bd6e5d87455ac4b1f047b5af9e24f1
I20260926 06:56:41.466804 21142 catalog_manager.cc:1347] Generated new cluster ID: 21466785b55445bcb9f8862211478414
I20260926 06:56:41.466872 21142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:41.475282 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:41.466608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42331 (local address 127.0.0.1:36191)
0926 06:56:41.466768 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:41.466777 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:41.469626 (+ 2849us) server_negotiation.cc:407] Connection header received
0926 06:56:41.469764 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:41.469771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:41.469831 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:41.469954 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:41.470199 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:41.471915 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:41.473034 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:41.473637 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:41.473731 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:41.474552 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:41.474556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:41.474556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:41.474786 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:41.474823 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:41.474851 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:41.474936 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:41.475046 (+ 110us) server_negotiation.cc:299] Negotiation successful
0926 06:56:41.475071 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260926 06:56:41.476224 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:41.481164 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:41.477787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43273 (local address 127.0.0.1:36191)
0926 06:56:41.477826 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:41.477828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:41.477837 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:56:41.477918 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:41.477919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:41.477929 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:41.477991 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:41.478130 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:41.478872 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:41.479820 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:41.480147 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:41.480219 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:41.480915 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:41.480917 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:41.480917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:41.480923 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:41.480938 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:41.480942 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:41.481007 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:41.481080 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 06:56:41.481092 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:56:41.490535 21142 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:41.491128 21142 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:41.493431 21142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b6347a292db42dfa1151f58c34403a0: Generated new TSK 0
I20260926 06:56:41.493602 21142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:41.496932 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 21066
W20260926 06:56:41.500874 21051 connection.cc:535] client connection to 127.0.0.1:36191 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36191): Connection reset by peer (error 104)
2026-09-26T06:56:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1322 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:41 dist-test-slave-ws74 krb5kdc[21165](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:41 dist-test-slave-ws74 krb5kdc[21165](info): set up 2 sockets
Sep 26 06:56:41 dist-test-slave-ws74 krb5kdc[21165](info): commencing operation
krb5kdc: starting...
W20260926 06:56:42.529266 767 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:42 dist-test-slave-ws74 krb5kdc[21165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:56:42.643735 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46699
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:42.723657 21178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:42.723896 21178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:42.723937 21178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:42.725069 21178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:42.725142 21178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:42.725167 21178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:42.725194 21178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:42.725221 21178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:42.726691 21178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46699
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:42.726955 21178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:42.727186 21178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:42.731817 21184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:42.731813 21186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:42.731851 21183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:42.732081 21178 server_base.cc:1034] running on GCE node
I20260926 06:56:42.732343 21178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:42.732578 21178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:42.733716 21178 hybrid_clock.cc:648] HybridClock initialized: now 1790405802733692 us; error 49 us; skew 500 ppm
Sep 26 06:56:42 dist-test-slave-ws74 krb5kdc[21165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405802, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:42.736131 21178 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:56:42.736641 21178 webserver.cc:460] Webserver started at http://127.0.0.1:44137/ using document root <none> and password file <none>
I20260926 06:56:42.736894 21178 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:42.736938 21178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:42.737083 21178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:42.738221 21178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "de2b99e3799c41099cc1adbf1ea16b0b"
format_stamp: "Formatted at 2026-09-26 06:56:42 on dist-test-slave-ws74"
server_key: "a6c89d210682ac1513ca9c3b1feab716"
server_key_iv: "bfa99b1d7f51608c07d8369ebef6d2c5"
server_key_version: "encryptionkey@0"
I20260926 06:56:42.738574 21178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "de2b99e3799c41099cc1adbf1ea16b0b"
format_stamp: "Formatted at 2026-09-26 06:56:42 on dist-test-slave-ws74"
server_key: "a6c89d210682ac1513ca9c3b1feab716"
server_key_iv: "bfa99b1d7f51608c07d8369ebef6d2c5"
server_key_version: "encryptionkey@0"
I20260926 06:56:42.739876 21178 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:56:42.740604 21193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:42.740780 21178 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:56:42.740880 21178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "de2b99e3799c41099cc1adbf1ea16b0b"
format_stamp: "Formatted at 2026-09-26 06:56:42 on dist-test-slave-ws74"
server_key: "a6c89d210682ac1513ca9c3b1feab716"
server_key_iv: "bfa99b1d7f51608c07d8369ebef6d2c5"
server_key_version: "encryptionkey@0"
I20260926 06:56:42.740960 21178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:42.744230 21178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:42.746064 21178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:42.746215 21178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:42.750191 21178 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46699
I20260926 06:56:42.750236 21245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46699 every 8 connection(s)
I20260926 06:56:42.750574 21178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:42.751489 21246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:42.753618 21246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b: Bootstrap starting.
I20260926 06:56:42.754117 21246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:42.754349 21246 log.cc:824] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:42.754954 21246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b: No bootstrap required, opened a new log
I20260926 06:56:42.756431 21246 raft_consensus.cc:348] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2b99e3799c41099cc1adbf1ea16b0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46699 } }
I20260926 06:56:42.756558 21246 raft_consensus.cc:374] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:42.756580 21246 raft_consensus.cc:729] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2b99e3799c41099cc1adbf1ea16b0b, State: Initialized, Role: FOLLOWER
I20260926 06:56:42.756709 21246 consensus_queue.cc:260] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2b99e3799c41099cc1adbf1ea16b0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46699 } }
I20260926 06:56:42.756791 21246 raft_consensus.cc:388] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:42.756815 21246 raft_consensus.cc:482] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:42.756847 21246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:42.757320 21246 raft_consensus.cc:504] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2b99e3799c41099cc1adbf1ea16b0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46699 } }
I20260926 06:56:42.757462 21246 leader_election.cc:302] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de2b99e3799c41099cc1adbf1ea16b0b; no voters:
I20260926 06:56:42.757586 21246 leader_election.cc:288] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:42.757664 21249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:42.757834 21246 sys_catalog.cc:564] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:42.757982 21249 raft_consensus.cc:686] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [term 1 LEADER]: Becoming Leader. State: Replica: de2b99e3799c41099cc1adbf1ea16b0b, State: Running, Role: LEADER
I20260926 06:56:42.758111 21249 consensus_queue.cc:237] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2b99e3799c41099cc1adbf1ea16b0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46699 } }
I20260926 06:56:42.758214 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 21178
I20260926 06:56:42.758338 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:42.758458 21250 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de2b99e3799c41099cc1adbf1ea16b0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2b99e3799c41099cc1adbf1ea16b0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46699 } } }
I20260926 06:56:42.758534 21250 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:42.758564 767 external_mini_cluster.cc:1428] Setting key 8ce2b70b2ca8863f39e0b61135c09d3c
I20260926 06:56:42.758700 21251 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader de2b99e3799c41099cc1adbf1ea16b0b. Latest consensus state: current_term: 1 leader_uuid: "de2b99e3799c41099cc1adbf1ea16b0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2b99e3799c41099cc1adbf1ea16b0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46699 } } }
I20260926 06:56:42.758752 21251 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:42.759210 21258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:42.759706 21258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:42.760782 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:42.759256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36305 (local address 127.0.0.1:46699)
0926 06:56:42.759410 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:42.759419 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:42.759447 (+ 28us) server_negotiation.cc:407] Connection header received
0926 06:56:42.759548 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:42.759553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:42.759575 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:42.759712 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:42.760379 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:42.760383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:42.760383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:42.760387 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:42.760435 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:42.760446 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:42.760523 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:42.760594 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 06:56:42.760635 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260926 06:56:42.761571 21258 catalog_manager.cc:1347] Generated new cluster ID: b1eec49ccfba4ab9ad6b8333a8acb83f
I20260926 06:56:42.761620 21258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:42.761665 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:42.764230 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:42.763240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35213 (local address 127.0.0.1:46699)
0926 06:56:42.763298 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:42.763299 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:42.763307 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:56:42.763401 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:42.763402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:42.763408 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:42.763476 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:42.764011 (+ 535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:42.764013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:42.764013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:42.764019 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:42.764034 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:42.764039 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:42.764099 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:42.764148 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 06:56:42.764161 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 06:56:42.772135 21258 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:42.772281 21258 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:42.775519 21258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de2b99e3799c41099cc1adbf1ea16b0b: Generated new TSK 0
I20260926 06:56:42.775691 21258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:42.783371 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 21178
2026-09-26T06:56:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1286 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:42 dist-test-slave-ws74 krb5kdc[21277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:42 dist-test-slave-ws74 krb5kdc[21277](info): set up 2 sockets
Sep 26 06:56:42 dist-test-slave-ws74 krb5kdc[21277](info): commencing operation
krb5kdc: starting...
W20260926 06:56:43.821353 767 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:43 dist-test-slave-ws74 krb5kdc[21277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:56:43.939364 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:44.021004 21290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:44.021211 21290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:44.021234 21290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:44.022428 21290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:44.022480 21290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:44.022495 21290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:44.022507 21290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:44.022519 21290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:44.023942 21290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:44.024130 21290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:44.024304 21290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:44.029150 21296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:44.029224 21295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:44.029400 21290 server_base.cc:1034] running on GCE node
W20260926 06:56:44.029145 21298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:44.029742 21290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:44.029992 21290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:44.031144 21290 hybrid_clock.cc:648] HybridClock initialized: now 1790405804031131 us; error 29 us; skew 500 ppm
Sep 26 06:56:44 dist-test-slave-ws74 krb5kdc[21277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405804, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:44.036684 21290 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:56:44.037195 21290 webserver.cc:460] Webserver started at http://127.0.0.1:34245/ using document root <none> and password file <none>
I20260926 06:56:44.037478 21290 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:44.037531 21290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:44.037664 21290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:44.038754 21290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "b483890e99f04000b76b0919ab1039b6"
format_stamp: "Formatted at 2026-09-26 06:56:44 on dist-test-slave-ws74"
server_key: "8cea14b25e9d817dd0dbe41331268699"
server_key_iv: "6ed953570c6b4549b9a59f32479cda50"
server_key_version: "encryptionkey@0"
I20260926 06:56:44.039114 21290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "b483890e99f04000b76b0919ab1039b6"
format_stamp: "Formatted at 2026-09-26 06:56:44 on dist-test-slave-ws74"
server_key: "8cea14b25e9d817dd0dbe41331268699"
server_key_iv: "6ed953570c6b4549b9a59f32479cda50"
server_key_version: "encryptionkey@0"
I20260926 06:56:44.040517 21290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:56:44.041261 21305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:44.041467 21290 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:44.041574 21290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "b483890e99f04000b76b0919ab1039b6"
format_stamp: "Formatted at 2026-09-26 06:56:44 on dist-test-slave-ws74"
server_key: "8cea14b25e9d817dd0dbe41331268699"
server_key_iv: "6ed953570c6b4549b9a59f32479cda50"
server_key_version: "encryptionkey@0"
I20260926 06:56:44.041659 21290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:44.051026 21290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:44.052788 21290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:44.052947 21290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:44.056900 21290 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44777
I20260926 06:56:44.056932 21357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44777 every 8 connection(s)
I20260926 06:56:44.057313 21290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:44.058148 21358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:44.060313 21358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6: Bootstrap starting.
I20260926 06:56:44.060823 21358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:44.061058 21358 log.cc:824] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:44.061728 21358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6: No bootstrap required, opened a new log
I20260926 06:56:44.063241 21358 raft_consensus.cc:348] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b483890e99f04000b76b0919ab1039b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44777 } }
I20260926 06:56:44.063395 21358 raft_consensus.cc:374] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:44.063418 21358 raft_consensus.cc:729] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b483890e99f04000b76b0919ab1039b6, State: Initialized, Role: FOLLOWER
I20260926 06:56:44.063555 21358 consensus_queue.cc:260] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b483890e99f04000b76b0919ab1039b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44777 } }
I20260926 06:56:44.063637 21358 raft_consensus.cc:388] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:44.063658 21358 raft_consensus.cc:482] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:44.063727 21358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:44.063863 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 21290
I20260926 06:56:44.063978 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:44.064150 767 external_mini_cluster.cc:1428] Setting key a6c03e9874b7ab57faf1ce391b0cacb3
I20260926 06:56:44.064427 21358 raft_consensus.cc:504] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b483890e99f04000b76b0919ab1039b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44777 } }
I20260926 06:56:44.064561 21358 leader_election.cc:302] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b483890e99f04000b76b0919ab1039b6; no voters:
I20260926 06:56:44.064702 21358 leader_election.cc:288] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:44.064790 21363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:44.064929 21363 raft_consensus.cc:686] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [term 1 LEADER]: Becoming Leader. State: Replica: b483890e99f04000b76b0919ab1039b6, State: Running, Role: LEADER
I20260926 06:56:44.065032 21358 sys_catalog.cc:564] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:44.065063 21363 consensus_queue.cc:237] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b483890e99f04000b76b0919ab1039b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44777 } }
I20260926 06:56:44.065523 21363 sys_catalog.cc:455] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b483890e99f04000b76b0919ab1039b6. Latest consensus state: current_term: 1 leader_uuid: "b483890e99f04000b76b0919ab1039b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b483890e99f04000b76b0919ab1039b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44777 } } }
I20260926 06:56:44.065551 21365 sys_catalog.cc:455] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b483890e99f04000b76b0919ab1039b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b483890e99f04000b76b0919ab1039b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44777 } } }
I20260926 06:56:44.065661 21365 sys_catalog.cc:458] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:44.065861 21363 sys_catalog.cc:458] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:44.066057 21368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:44.066659 21368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:44.068476 21368 catalog_manager.cc:1347] Generated new cluster ID: 783e457e89644d94b7eff94bf7350683
I20260926 06:56:44.068531 21368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:56:44 dist-test-slave-ws74 krb5kdc[21277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 06:56:44.076072 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:44.064586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43531 (local address 127.0.0.1:44777)
0926 06:56:44.064766 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:44.064776 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:44.064816 (+ 40us) server_negotiation.cc:407] Connection header received
0926 06:56:44.064938 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:44.064945 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:44.065000 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:44.065962 (+ 962us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:44.066091 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:44.067712 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:44.068882 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:44.069580 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:44.069695 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:44.072468 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:44.072472 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:44.072472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:44.072483 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:44.074444 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:44.074848 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:44.074850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:44.074851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:44.074974 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:44.075191 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:44.075192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:44.075192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:44.075470 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:44.075613 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:44.075722 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 06:56:44.075846 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260926 06:56:44.076964 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:44.085438 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:44.078415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38523 (local address 127.0.0.1:44777)
0926 06:56:44.078470 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:44.078472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:44.078697 (+ 225us) server_negotiation.cc:407] Connection header received
0926 06:56:44.078755 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:44.078756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:44.078766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:44.078818 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:44.079404 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:44.080111 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:44.081102 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:44.081409 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:44.081461 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:44.082767 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:44.082768 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:44.082769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:44.082774 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:44.084291 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:44.084662 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:44.084663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:44.084663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:44.084714 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:44.084913 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:44.084914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:44.084914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:44.085027 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:44.085120 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:44.085244 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 06:56:44.085317 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 06:56:44.094101 21368 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:44.094622 21368 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:44.099594 21368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b483890e99f04000b76b0919ab1039b6: Generated new TSK 0
I20260926 06:56:44.099779 21368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:44.103785 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 21290
2026-09-26T06:56:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:44 dist-test-slave-ws74 krb5kdc[21389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:44 dist-test-slave-ws74 krb5kdc[21389](info): set up 2 sockets
Sep 26 06:56:44 dist-test-slave-ws74 krb5kdc[21389](info): commencing operation
krb5kdc: starting...
W20260926 06:56:45.161557 767 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:45 dist-test-slave-ws74 krb5kdc[21389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:56:45.280422 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37891
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:45.358959 21402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:45.359150 21402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:45.359172 21402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:45.360297 21402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:45.360339 21402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:45.360354 21402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:45.360392 21402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:45.360423 21402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:45.361855 21402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:45.362093 21402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:45.362320 21402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:45.367044 21407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:45.367072 21410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:45.367061 21408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:45.367096 21402 server_base.cc:1034] running on GCE node
I20260926 06:56:45.367456 21402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:45.367777 21402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:45.368952 21402 hybrid_clock.cc:648] HybridClock initialized: now 1790405805368930 us; error 38 us; skew 500 ppm
Sep 26 06:56:45 dist-test-slave-ws74 krb5kdc[21389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405805, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:45.371469 21402 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:56:45.371941 21402 webserver.cc:460] Webserver started at http://127.0.0.1:39829/ using document root <none> and password file <none>
I20260926 06:56:45.372139 21402 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:45.372179 21402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:45.372320 21402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:45.373291 21402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "36acd89d46424eed94033b08ecd6c027"
format_stamp: "Formatted at 2026-09-26 06:56:45 on dist-test-slave-ws74"
server_key: "dae6f0c4687103340db8538f21e3b562"
server_key_iv: "801cfea05b75c7421de96a0691320cd5"
server_key_version: "encryptionkey@0"
I20260926 06:56:45.373679 21402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "36acd89d46424eed94033b08ecd6c027"
format_stamp: "Formatted at 2026-09-26 06:56:45 on dist-test-slave-ws74"
server_key: "dae6f0c4687103340db8538f21e3b562"
server_key_iv: "801cfea05b75c7421de96a0691320cd5"
server_key_version: "encryptionkey@0"
I20260926 06:56:45.374951 21402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:45.375612 21417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:45.375775 21402 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:45.375882 21402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "36acd89d46424eed94033b08ecd6c027"
format_stamp: "Formatted at 2026-09-26 06:56:45 on dist-test-slave-ws74"
server_key: "dae6f0c4687103340db8538f21e3b562"
server_key_iv: "801cfea05b75c7421de96a0691320cd5"
server_key_version: "encryptionkey@0"
I20260926 06:56:45.375957 21402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:45.394877 21402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:45.396600 21402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:45.396759 21402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:45.400508 21402 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37891
I20260926 06:56:45.400548 21469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37891 every 8 connection(s)
I20260926 06:56:45.400916 21402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:45.401841 21470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:45.404023 21470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027: Bootstrap starting.
I20260926 06:56:45.404588 21470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:45.404868 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 21402
I20260926 06:56:45.404886 21470 log.cc:824] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:45.404999 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:45.405212 767 external_mini_cluster.cc:1428] Setting key f0ccdaee425b291e279279a50bc99f48
I20260926 06:56:45.405778 21470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027: No bootstrap required, opened a new log
I20260926 06:56:45.408468 21470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36acd89d46424eed94033b08ecd6c027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37891 } }
I20260926 06:56:45.408645 21470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:45.408675 21470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36acd89d46424eed94033b08ecd6c027, State: Initialized, Role: FOLLOWER
I20260926 06:56:45.408782 21470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36acd89d46424eed94033b08ecd6c027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37891 } }
I20260926 06:56:45.408844 21470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:45.408872 21470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:45.408921 21470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:45.409523 21470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36acd89d46424eed94033b08ecd6c027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37891 } }
I20260926 06:56:45.409611 21470 leader_election.cc:302] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36acd89d46424eed94033b08ecd6c027; no voters:
I20260926 06:56:45.409782 21470 leader_election.cc:288] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:45.409807 21475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:45.409969 21475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [term 1 LEADER]: Becoming Leader. State: Replica: 36acd89d46424eed94033b08ecd6c027, State: Running, Role: LEADER
I20260926 06:56:45.410090 21470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:45.410110 21475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36acd89d46424eed94033b08ecd6c027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37891 } }
I20260926 06:56:45.410429 21475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36acd89d46424eed94033b08ecd6c027. Latest consensus state: current_term: 1 leader_uuid: "36acd89d46424eed94033b08ecd6c027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36acd89d46424eed94033b08ecd6c027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37891 } } }
I20260926 06:56:45.410540 21475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:45.410447 21476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36acd89d46424eed94033b08ecd6c027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36acd89d46424eed94033b08ecd6c027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37891 } } }
I20260926 06:56:45.410806 21476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:45.410982 21482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:45.412066 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:45.405750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53679 (local address 127.0.0.1:37891)
0926 06:56:45.405943 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:45.405952 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:45.405988 (+ 36us) server_negotiation.cc:407] Connection header received
0926 06:56:45.406982 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:45.406991 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:45.407060 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:45.407211 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:45.407457 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:45.408891 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:45.409844 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:45.410472 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:45.410622 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:45.411363 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:45.411367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:45.411367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:45.411571 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:45.411600 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:45.411611 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:45.411659 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:45.411773 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 06:56:45.411839 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":77,"threads_started":1}
I20260926 06:56:45.412106 21482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:45.413005 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:45.413623 21482 catalog_manager.cc:1347] Generated new cluster ID: 75447ea8d61849c3b651a6f0201fae90
I20260926 06:56:45.413664 21482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:45.417801 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:45.414498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36865 (local address 127.0.0.1:37891)
0926 06:56:45.414531 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:45.414533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:45.414608 (+ 75us) server_negotiation.cc:407] Connection header received
0926 06:56:45.414660 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:45.414661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:45.414670 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:45.414737 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:45.414873 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:45.415599 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:45.416568 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:45.416866 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:45.416940 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:45.417570 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:45.417571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:45.417571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:45.417576 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:45.417591 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:45.417595 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:45.417656 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:45.417712 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 06:56:45.417731 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:56:45.427125 21482 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:45.427630 21482 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:45.430263 21482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36acd89d46424eed94033b08ecd6c027: Generated new TSK 0
I20260926 06:56:45.430429 21482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:45.432073 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 21402
2026-09-26T06:56:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:45 dist-test-slave-ws74 krb5kdc[21501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:45 dist-test-slave-ws74 krb5kdc[21501](info): set up 2 sockets
Sep 26 06:56:45 dist-test-slave-ws74 krb5kdc[21501](info): commencing operation
krb5kdc: starting...
W20260926 06:56:46.453504 767 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:46 dist-test-slave-ws74 krb5kdc[21501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:56:46.566339 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33043
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:46.644588 21514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:46.644789 21514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:46.644811 21514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:46.645987 21514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:46.646030 21514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:46.646044 21514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:46.646057 21514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:46.646116 21514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:46.647567 21514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:46.647801 21514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:46.648023 21514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:46.653189 21519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:46.653194 21522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:46.653235 21514 server_base.cc:1034] running on GCE node
W20260926 06:56:46.653189 21520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:46.653658 21514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:46.653959 21514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:46.655128 21514 hybrid_clock.cc:648] HybridClock initialized: now 1790405806655102 us; error 56 us; skew 500 ppm
Sep 26 06:56:46 dist-test-slave-ws74 krb5kdc[21501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405806, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:46.657781 21514 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:56:46.658264 21514 webserver.cc:460] Webserver started at http://127.0.0.1:37717/ using document root <none> and password file <none>
I20260926 06:56:46.658500 21514 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:46.658567 21514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:46.658716 21514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:46.659754 21514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "3d0bdc4750354e1789a8e1d5d10ee96a"
format_stamp: "Formatted at 2026-09-26 06:56:46 on dist-test-slave-ws74"
server_key: "76696bc1d5f9ea8c3eb06209a9383601"
server_key_iv: "4433cf8c1b6bc26636d192a9a2be615d"
server_key_version: "encryptionkey@0"
I20260926 06:56:46.660162 21514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "3d0bdc4750354e1789a8e1d5d10ee96a"
format_stamp: "Formatted at 2026-09-26 06:56:46 on dist-test-slave-ws74"
server_key: "76696bc1d5f9ea8c3eb06209a9383601"
server_key_iv: "4433cf8c1b6bc26636d192a9a2be615d"
server_key_version: "encryptionkey@0"
I20260926 06:56:46.661444 21514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:56:46.662106 21529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:46.662261 21514 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:46.662362 21514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "3d0bdc4750354e1789a8e1d5d10ee96a"
format_stamp: "Formatted at 2026-09-26 06:56:46 on dist-test-slave-ws74"
server_key: "76696bc1d5f9ea8c3eb06209a9383601"
server_key_iv: "4433cf8c1b6bc26636d192a9a2be615d"
server_key_version: "encryptionkey@0"
I20260926 06:56:46.662433 21514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:46.665715 21514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:46.667879 21514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:46.668032 21514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:46.672019 21514 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33043
I20260926 06:56:46.672053 21581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33043 every 8 connection(s)
I20260926 06:56:46.672359 21514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:46.673288 21582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:46.675621 21582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a: Bootstrap starting.
I20260926 06:56:46.676128 21582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:46.676355 21582 log.cc:824] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:46.676970 21582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a: No bootstrap required, opened a new log
I20260926 06:56:46.678588 21582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0bdc4750354e1789a8e1d5d10ee96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } }
I20260926 06:56:46.678720 21582 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:46.678742 21582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0bdc4750354e1789a8e1d5d10ee96a, State: Initialized, Role: FOLLOWER
I20260926 06:56:46.678872 21582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0bdc4750354e1789a8e1d5d10ee96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } }
I20260926 06:56:46.678951 21582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:46.678974 21582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:46.679003 21582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:46.679468 21582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0bdc4750354e1789a8e1d5d10ee96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } }
I20260926 06:56:46.679558 21582 leader_election.cc:302] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d0bdc4750354e1789a8e1d5d10ee96a; no voters:
I20260926 06:56:46.679674 21582 leader_election.cc:288] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:46.679718 21585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:46.679877 21585 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [term 1 LEADER]: Becoming Leader. State: Replica: 3d0bdc4750354e1789a8e1d5d10ee96a, State: Running, Role: LEADER
I20260926 06:56:46.679946 21582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:46.680019 21585 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0bdc4750354e1789a8e1d5d10ee96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } }
I20260926 06:56:46.680415 21586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d0bdc4750354e1789a8e1d5d10ee96a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0bdc4750354e1789a8e1d5d10ee96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } } }
I20260926 06:56:46.680536 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 21514
I20260926 06:56:46.680547 21586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:46.680652 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:46.680419 21587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d0bdc4750354e1789a8e1d5d10ee96a. Latest consensus state: current_term: 1 leader_uuid: "3d0bdc4750354e1789a8e1d5d10ee96a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0bdc4750354e1789a8e1d5d10ee96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } } }
I20260926 06:56:46.680764 21587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:46.680807 21589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:46.680873 767 external_mini_cluster.cc:1428] Setting key 5c4341ebffd3c0a6149a482383121c2b
I20260926 06:56:46.681360 21589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:46.686545 21589 catalog_manager.cc:1347] Generated new cluster ID: b8cd9d25a4404f5fba88fe2a750ff357
I20260926 06:56:46.686607 21589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:46.691571 21589 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:46.691758 21589 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:46.691934 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:46.681611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47053 (local address 127.0.0.1:33043)
0926 06:56:46.685649 (+ 4038us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:46.685657 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:46.685687 (+ 30us) server_negotiation.cc:407] Connection header received
0926 06:56:46.685798 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:46.685805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:46.685834 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:46.685989 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:46.691288 (+ 5299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:46.691291 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:46.691291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:46.691295 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:46.691354 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:46.691365 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:46.691472 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:46.691642 (+ 170us) server_negotiation.cc:299] Negotiation successful
0926 06:56:46.691687 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3989,"thread_start_us":69,"threads_started":1}
I20260926 06:56:46.692755 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:46.695343 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:46.694210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35793 (local address 127.0.0.1:33043)
0926 06:56:46.694248 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:46.694250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:46.694374 (+ 124us) server_negotiation.cc:407] Connection header received
0926 06:56:46.694448 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:46.694449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:46.694455 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:46.694516 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:46.695130 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:46.695131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:46.695131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:46.695135 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:46.695149 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:46.695153 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:46.695199 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:46.695248 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 06:56:46.695261 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:56:46.699296 21589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d0bdc4750354e1789a8e1d5d10ee96a: Generated new TSK 0
I20260926 06:56:46.699453 21589 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:46.705160 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 21514
2026-09-26T06:56:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1273 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:46 dist-test-slave-ws74 krb5kdc[21613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:46 dist-test-slave-ws74 krb5kdc[21613](info): set up 2 sockets
Sep 26 06:56:46 dist-test-slave-ws74 krb5kdc[21613](info): commencing operation
krb5kdc: starting...
W20260926 06:56:47.729477 767 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:47 dist-test-slave-ws74 krb5kdc[21613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:56:47.848012 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:47.928357 21626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:47.928570 21626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:47.928593 21626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:47.929726 21626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:47.929768 21626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:47.929782 21626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:47.929795 21626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:47.929807 21626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:47.931185 21626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:47.931372 21626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:47.931540 21626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:47.936487 21631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:47.936532 21632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:47.936555 21626 server_base.cc:1034] running on GCE node
W20260926 06:56:47.936506 21634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:47.936975 21626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:47.937238 21626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:47.938465 21626 hybrid_clock.cc:648] HybridClock initialized: now 1790405807938458 us; error 43 us; skew 500 ppm
Sep 26 06:56:47 dist-test-slave-ws74 krb5kdc[21613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405807, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:47.941023 21626 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:56:47.941538 21626 webserver.cc:460] Webserver started at http://127.0.0.1:45867/ using document root <none> and password file <none>
I20260926 06:56:47.941773 21626 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:47.941818 21626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:47.941984 21626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:47.943022 21626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "abdbc5c61d7c49b4b8e1fde253e33eff"
format_stamp: "Formatted at 2026-09-26 06:56:47 on dist-test-slave-ws74"
server_key: "454f905a74a92aa3899cde4649ed8dff"
server_key_iv: "f2731b43fe74fe4b87ab002cf543a336"
server_key_version: "encryptionkey@0"
I20260926 06:56:47.943384 21626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "abdbc5c61d7c49b4b8e1fde253e33eff"
format_stamp: "Formatted at 2026-09-26 06:56:47 on dist-test-slave-ws74"
server_key: "454f905a74a92aa3899cde4649ed8dff"
server_key_iv: "f2731b43fe74fe4b87ab002cf543a336"
server_key_version: "encryptionkey@0"
I20260926 06:56:47.944837 21626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:56:47.945657 21641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:47.945849 21626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:47.945953 21626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "abdbc5c61d7c49b4b8e1fde253e33eff"
format_stamp: "Formatted at 2026-09-26 06:56:47 on dist-test-slave-ws74"
server_key: "454f905a74a92aa3899cde4649ed8dff"
server_key_iv: "f2731b43fe74fe4b87ab002cf543a336"
server_key_version: "encryptionkey@0"
I20260926 06:56:47.946036 21626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:47.955778 21626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:47.958029 21626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:47.958189 21626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:47.962270 21626 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40709
I20260926 06:56:47.962297 21693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40709 every 8 connection(s)
I20260926 06:56:47.962661 21626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:47.963521 21694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:47.965727 21694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff: Bootstrap starting.
I20260926 06:56:47.966323 21694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:47.966615 21694 log.cc:824] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:47.967247 21694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff: No bootstrap required, opened a new log
I20260926 06:56:47.968767 21694 raft_consensus.cc:348] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdbc5c61d7c49b4b8e1fde253e33eff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40709 } }
I20260926 06:56:47.968901 21694 raft_consensus.cc:374] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:47.968948 21694 raft_consensus.cc:729] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abdbc5c61d7c49b4b8e1fde253e33eff, State: Initialized, Role: FOLLOWER
I20260926 06:56:47.969111 21694 consensus_queue.cc:260] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdbc5c61d7c49b4b8e1fde253e33eff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40709 } }
I20260926 06:56:47.969172 21694 raft_consensus.cc:388] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:47.969192 21694 raft_consensus.cc:482] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:47.969257 21694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:47.969798 21694 raft_consensus.cc:504] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdbc5c61d7c49b4b8e1fde253e33eff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40709 } }
I20260926 06:56:47.969916 21694 leader_election.cc:302] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abdbc5c61d7c49b4b8e1fde253e33eff; no voters:
I20260926 06:56:47.970099 21694 leader_election.cc:288] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:47.970121 21697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:47.970290 21697 raft_consensus.cc:686] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [term 1 LEADER]: Becoming Leader. State: Replica: abdbc5c61d7c49b4b8e1fde253e33eff, State: Running, Role: LEADER
I20260926 06:56:47.970412 21694 sys_catalog.cc:564] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:47.970429 21697 consensus_queue.cc:237] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdbc5c61d7c49b4b8e1fde253e33eff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40709 } }
I20260926 06:56:47.970890 21699 sys_catalog.cc:455] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [sys.catalog]: SysCatalogTable state changed. Reason: New leader abdbc5c61d7c49b4b8e1fde253e33eff. Latest consensus state: current_term: 1 leader_uuid: "abdbc5c61d7c49b4b8e1fde253e33eff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdbc5c61d7c49b4b8e1fde253e33eff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40709 } } }
I20260926 06:56:47.971010 21698 sys_catalog.cc:455] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abdbc5c61d7c49b4b8e1fde253e33eff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdbc5c61d7c49b4b8e1fde253e33eff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40709 } } }
I20260926 06:56:47.971041 21699 sys_catalog.cc:458] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:47.971141 21698 sys_catalog.cc:458] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:47.971473 21706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:47.972049 21706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:47.972452 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 21626
I20260926 06:56:47.972573 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:47.972785 767 external_mini_cluster.cc:1428] Setting key 6f65ba705e830089a3b6f46c63c7a7d5
I20260926 06:56:47.974542 21706 catalog_manager.cc:1347] Generated new cluster ID: d7f265a331d64554ba70361cd92103c2
I20260926 06:56:47.974619 21706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:56:47 dist-test-slave-ws74 krb5kdc[21613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 06:56:47.988610 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:47.973363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:51167 (local address 127.0.0.1:40709)
0926 06:56:47.973637 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:47.973647 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:47.977868 (+ 4221us) server_negotiation.cc:407] Connection header received
0926 06:56:47.978001 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:47.978009 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:47.978067 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:47.978221 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:47.978933 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:47.980469 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:47.981477 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:47.982102 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:47.982219 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:47.984981 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:47.984986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:47.984986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:47.984997 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:47.986921 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:47.987382 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:47.987384 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:47.987385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:47.987484 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:47.987710 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:47.987710 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:47.987711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:47.988013 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:47.988157 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:47.988294 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 06:56:47.988393 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":80,"threads_started":1}
I20260926 06:56:47.989843 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:47.990727 21706 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:47.991430 21706 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:47.998492 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:47.991306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:34795 (local address 127.0.0.1:40709)
0926 06:56:47.991348 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:47.991350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:47.991435 (+ 85us) server_negotiation.cc:407] Connection header received
0926 06:56:47.991487 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:47.991488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:47.991521 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:47.991583 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:47.992189 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:47.992968 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:47.994041 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:47.994379 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:47.994439 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:47.995783 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:47.995785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:47.995785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:47.995791 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:47.997296 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:47.997729 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:47.997730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:47.997730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:47.997789 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:47.998020 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:47.998021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:47.998021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:47.998125 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:47.998226 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:47.998352 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 06:56:47.998407 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 06:56:47.999750 21706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abdbc5c61d7c49b4b8e1fde253e33eff: Generated new TSK 0
I20260926 06:56:47.999912 21706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:48.002967 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 21626
W20260926 06:56:48.007225 21611 connection.cc:535] client connection to 127.0.0.1:40709 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40709): Connection reset by peer (error 104)
2026-09-26T06:56:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:48 dist-test-slave-ws74 krb5kdc[21725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:48 dist-test-slave-ws74 krb5kdc[21725](info): set up 2 sockets
Sep 26 06:56:48 dist-test-slave-ws74 krb5kdc[21725](info): commencing operation
krb5kdc: starting...
W20260926 06:56:49.053376 767 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:49 dist-test-slave-ws74 krb5kdc[21725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:56:49.170065 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34395
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:49.249184 21738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:49.249397 21738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:49.249421 21738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:49.250548 21738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:49.250590 21738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:49.250605 21738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:49.250617 21738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:49.250629 21738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:49.251982 21738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:49.252161 21738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:49.252316 21738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:49.257004 21744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:49.257064 21743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:49.257011 21746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:49.257148 21738 server_base.cc:1034] running on GCE node
I20260926 06:56:49.257514 21738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:49.257788 21738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:49.258958 21738 hybrid_clock.cc:648] HybridClock initialized: now 1790405809258945 us; error 30 us; skew 500 ppm
Sep 26 06:56:49 dist-test-slave-ws74 krb5kdc[21725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405809, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:49.264649 21738 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:56:49.265131 21738 webserver.cc:460] Webserver started at http://127.0.0.1:42451/ using document root <none> and password file <none>
I20260926 06:56:49.265352 21738 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:49.265467 21738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:49.265621 21738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:49.266675 21738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "de296a86e8ac42ab88fbf01bbf46edf0"
format_stamp: "Formatted at 2026-09-26 06:56:49 on dist-test-slave-ws74"
server_key: "143a62bde5363439ad04018934472353"
server_key_iv: "9c2abb9513d87b0fa2f71a9bdab26fba"
server_key_version: "encryptionkey@0"
I20260926 06:56:49.267036 21738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "de296a86e8ac42ab88fbf01bbf46edf0"
format_stamp: "Formatted at 2026-09-26 06:56:49 on dist-test-slave-ws74"
server_key: "143a62bde5363439ad04018934472353"
server_key_iv: "9c2abb9513d87b0fa2f71a9bdab26fba"
server_key_version: "encryptionkey@0"
I20260926 06:56:49.268332 21738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:56:49.269043 21753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:49.269220 21738 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:56:49.269316 21738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "de296a86e8ac42ab88fbf01bbf46edf0"
format_stamp: "Formatted at 2026-09-26 06:56:49 on dist-test-slave-ws74"
server_key: "143a62bde5363439ad04018934472353"
server_key_iv: "9c2abb9513d87b0fa2f71a9bdab26fba"
server_key_version: "encryptionkey@0"
I20260926 06:56:49.269435 21738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:49.277927 21738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:49.279599 21738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:49.279778 21738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:49.283825 21738 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34395
I20260926 06:56:49.283857 21805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34395 every 8 connection(s)
I20260926 06:56:49.284191 21738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:49.284967 21806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:49.287190 21806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0: Bootstrap starting.
I20260926 06:56:49.287758 21806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:49.288038 21806 log.cc:824] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:49.288970 21806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0: No bootstrap required, opened a new log
I20260926 06:56:49.290683 21806 raft_consensus.cc:348] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de296a86e8ac42ab88fbf01bbf46edf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34395 } }
I20260926 06:56:49.290815 21806 raft_consensus.cc:374] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:49.290840 21806 raft_consensus.cc:729] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de296a86e8ac42ab88fbf01bbf46edf0, State: Initialized, Role: FOLLOWER
I20260926 06:56:49.291018 21806 consensus_queue.cc:260] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de296a86e8ac42ab88fbf01bbf46edf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34395 } }
I20260926 06:56:49.291096 21806 raft_consensus.cc:388] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:49.291141 21806 raft_consensus.cc:482] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:49.291189 21806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:49.291693 21806 raft_consensus.cc:504] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de296a86e8ac42ab88fbf01bbf46edf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34395 } }
I20260926 06:56:49.291805 21806 leader_election.cc:302] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de296a86e8ac42ab88fbf01bbf46edf0; no voters:
I20260926 06:56:49.291980 21806 leader_election.cc:288] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:49.292024 21809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:49.292210 21809 raft_consensus.cc:686] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [term 1 LEADER]: Becoming Leader. State: Replica: de296a86e8ac42ab88fbf01bbf46edf0, State: Running, Role: LEADER
I20260926 06:56:49.292232 21806 sys_catalog.cc:564] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:49.292295 21809 consensus_queue.cc:237] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de296a86e8ac42ab88fbf01bbf46edf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34395 } }
I20260926 06:56:49.292757 21810 sys_catalog.cc:455] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de296a86e8ac42ab88fbf01bbf46edf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de296a86e8ac42ab88fbf01bbf46edf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34395 } } }
I20260926 06:56:49.292770 21811 sys_catalog.cc:455] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de296a86e8ac42ab88fbf01bbf46edf0. Latest consensus state: current_term: 1 leader_uuid: "de296a86e8ac42ab88fbf01bbf46edf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de296a86e8ac42ab88fbf01bbf46edf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34395 } } }
I20260926 06:56:49.292927 21810 sys_catalog.cc:458] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:49.292946 21811 sys_catalog.cc:458] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:49.293339 21814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:49.293879 21814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:49.294373 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 21738
I20260926 06:56:49.294499 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:49.294683 767 external_mini_cluster.cc:1428] Setting key 3e104897cf1c1e13872e2ba31e6d0979
I20260926 06:56:49.295869 21814 catalog_manager.cc:1347] Generated new cluster ID: b073326cb6ab41bc824372f01a2aa728
I20260926 06:56:49.295928 21814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:49.300671 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:49.295126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:56597 (local address 127.0.0.1:34395)
0926 06:56:49.295303 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:49.295311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:49.295392 (+ 81us) server_negotiation.cc:407] Connection header received
0926 06:56:49.295532 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:49.295537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:49.295580 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:49.295717 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:49.295948 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:49.297340 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:49.298527 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:49.299113 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:49.299205 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:49.299946 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:49.299950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:49.299950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:49.300150 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:49.300185 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:49.300199 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:49.300288 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:49.300453 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 06:56:49.300479 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260926 06:56:49.301641 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:49.306584 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:49.303154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:40939 (local address 127.0.0.1:34395)
0926 06:56:49.303184 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:49.303186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:49.303303 (+ 117us) server_negotiation.cc:407] Connection header received
0926 06:56:49.303373 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:49.303374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:49.303384 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:49.303438 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:49.303570 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:49.304258 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:49.305279 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:49.305605 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:49.305665 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:49.306340 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:49.306342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:49.306342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:49.306346 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:49.306359 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:49.306363 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:49.306415 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:49.306501 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 06:56:49.306513 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260926 06:56:49.339277 21814 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:49.339869 21814 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:49.346123 21814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de296a86e8ac42ab88fbf01bbf46edf0: Generated new TSK 0
I20260926 06:56:49.346333 21814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:49.353645 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 21738
W20260926 06:56:49.358132 21723 connection.cc:535] client connection to 127.0.0.1:34395 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34395): Connection reset by peer (error 104)
2026-09-26T06:56:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:49 dist-test-slave-ws74 krb5kdc[21837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:49 dist-test-slave-ws74 krb5kdc[21837](info): set up 2 sockets
Sep 26 06:56:49 dist-test-slave-ws74 krb5kdc[21837](info): commencing operation
krb5kdc: starting...
W20260926 06:56:50.437510 767 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:50 dist-test-slave-ws74 krb5kdc[21837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:56:50.552064 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44487
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:50.631157 21850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:50.631394 21850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:50.631433 21850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:50.632579 21850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:50.632651 21850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:50.632678 21850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:50.632706 21850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:50.632733 21850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:50.634243 21850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:50.634511 21850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:50.634727 21850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:50.639627 21856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:50.639632 21855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:50.639777 21850 server_base.cc:1034] running on GCE node
W20260926 06:56:50.639632 21858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:50.640133 21850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:50.640384 21850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:50.641497 21850 hybrid_clock.cc:648] HybridClock initialized: now 1790405810641479 us; error 28 us; skew 500 ppm
Sep 26 06:56:50 dist-test-slave-ws74 krb5kdc[21837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405810, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:50.644053 21850 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:56:50.644520 21850 webserver.cc:460] Webserver started at http://127.0.0.1:35857/ using document root <none> and password file <none>
I20260926 06:56:50.644743 21850 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:50.644809 21850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:50.644939 21850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:50.646023 21850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "ec79dd2647614751a826a538dcc67385"
format_stamp: "Formatted at 2026-09-26 06:56:50 on dist-test-slave-ws74"
server_key: "e64cbf5052a01d442f1d3e648d3fb96a"
server_key_iv: "49ded1a04cc632b140c6fd7a0339c6ac"
server_key_version: "encryptionkey@0"
I20260926 06:56:50.646373 21850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "ec79dd2647614751a826a538dcc67385"
format_stamp: "Formatted at 2026-09-26 06:56:50 on dist-test-slave-ws74"
server_key: "e64cbf5052a01d442f1d3e648d3fb96a"
server_key_iv: "49ded1a04cc632b140c6fd7a0339c6ac"
server_key_version: "encryptionkey@0"
I20260926 06:56:50.647712 21850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:56:50.648455 21865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:50.648653 21850 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:50.648756 21850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "ec79dd2647614751a826a538dcc67385"
format_stamp: "Formatted at 2026-09-26 06:56:50 on dist-test-slave-ws74"
server_key: "e64cbf5052a01d442f1d3e648d3fb96a"
server_key_iv: "49ded1a04cc632b140c6fd7a0339c6ac"
server_key_version: "encryptionkey@0"
I20260926 06:56:50.648836 21850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:50.652011 21850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:50.653909 21850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:50.654059 21850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:50.658223 21850 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44487
I20260926 06:56:50.658272 21917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44487 every 8 connection(s)
I20260926 06:56:50.658595 21850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:50.659533 21918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:50.661753 21918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385: Bootstrap starting.
I20260926 06:56:50.662319 21918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:50.662606 21918 log.cc:824] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:50.663247 21918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385: No bootstrap required, opened a new log
I20260926 06:56:50.664825 21918 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec79dd2647614751a826a538dcc67385" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44487 } }
I20260926 06:56:50.664985 21918 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:50.665061 21918 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec79dd2647614751a826a538dcc67385, State: Initialized, Role: FOLLOWER
I20260926 06:56:50.665251 21918 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec79dd2647614751a826a538dcc67385" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44487 } }
I20260926 06:56:50.665359 21918 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:50.665465 21918 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:50.665522 21918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:50.666046 21918 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec79dd2647614751a826a538dcc67385" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44487 } }
I20260926 06:56:50.666179 21918 leader_election.cc:302] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec79dd2647614751a826a538dcc67385; no voters:
I20260926 06:56:50.666312 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 21850
I20260926 06:56:50.666344 21918 leader_election.cc:288] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:50.666419 21921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:50.666448 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:50.666569 21921 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [term 1 LEADER]: Becoming Leader. State: Replica: ec79dd2647614751a826a538dcc67385, State: Running, Role: LEADER
I20260926 06:56:50.666615 21918 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:50.666674 767 external_mini_cluster.cc:1428] Setting key cc66957a788a376e0537144ea7159340
I20260926 06:56:50.666707 21921 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec79dd2647614751a826a538dcc67385" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44487 } }
I20260926 06:56:50.667040 21921 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec79dd2647614751a826a538dcc67385. Latest consensus state: current_term: 1 leader_uuid: "ec79dd2647614751a826a538dcc67385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec79dd2647614751a826a538dcc67385" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44487 } } }
I20260926 06:56:50.667148 21921 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:50.667273 21923 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec79dd2647614751a826a538dcc67385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec79dd2647614751a826a538dcc67385" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44487 } } }
I20260926 06:56:50.667342 21923 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:50.667402 21927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:50.667994 21927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:50.669406 21928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:50.667276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:37113 (local address 127.0.0.1:44487)
0926 06:56:50.667818 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:50.667826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:50.667863 (+ 37us) server_negotiation.cc:407] Connection header received
0926 06:56:50.667978 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:50.667984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:50.668032 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:50.668492 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:50.668905 (+ 413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:50.668908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:50.668908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:50.668912 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:50.668950 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:50.668963 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:50.669041 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:50.669127 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 06:56:50.669170 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":70,"threads_started":1}
I20260926 06:56:50.669795 21927 catalog_manager.cc:1347] Generated new cluster ID: b43e34ef78f5459282e33dd0b7b3c177
I20260926 06:56:50.669839 21927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:50.670265 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:50.672703 21928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:50.671573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:41965 (local address 127.0.0.1:44487)
0926 06:56:50.671610 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:50.671612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:50.671724 (+ 112us) server_negotiation.cc:407] Connection header received
0926 06:56:50.671804 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:50.671805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:50.671809 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:50.671853 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:50.672447 (+ 594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:50.672448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:50.672448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:50.672452 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:50.672466 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:50.672470 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:50.672550 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:50.672607 (+ 57us) server_negotiation.cc:299] Negotiation successful
0926 06:56:50.672620 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:56:50.701774 21927 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:50.701982 21927 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:50.706744 21927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec79dd2647614751a826a538dcc67385: Generated new TSK 0
I20260926 06:56:50.706956 21927 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:50.715742 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 21850
2026-09-26T06:56:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1362 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:50 dist-test-slave-ws74 krb5kdc[21949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:50 dist-test-slave-ws74 krb5kdc[21949](info): set up 2 sockets
Sep 26 06:56:50 dist-test-slave-ws74 krb5kdc[21949](info): commencing operation
krb5kdc: starting...
W20260926 06:56:51.741562 767 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:51 dist-test-slave-ws74 krb5kdc[21949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:56:51.854280 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43539
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:51.932543 21962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:51.932731 21962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:51.932753 21962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:51.933983 21962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:51.934026 21962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:51.934041 21962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:51.934053 21962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:51.934101 21962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:51.935531 21962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43539
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:51.935773 21962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:51.935984 21962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:51.940652 21970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:51.940706 21962 server_base.cc:1034] running on GCE node
W20260926 06:56:51.940682 21968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:51.940749 21967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:51.941069 21962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:51.941336 21962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:51.942525 21962 hybrid_clock.cc:648] HybridClock initialized: now 1790405811942502 us; error 34 us; skew 500 ppm
Sep 26 06:56:51 dist-test-slave-ws74 krb5kdc[21949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405811, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:51.944876 21962 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:56:51.945273 21962 webserver.cc:460] Webserver started at http://127.0.0.1:46261/ using document root <none> and password file <none>
I20260926 06:56:51.945526 21962 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:51.945595 21962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:51.945746 21962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:51.946717 21962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "59096d362b4e4ed4bf52ea7286c8575c"
format_stamp: "Formatted at 2026-09-26 06:56:51 on dist-test-slave-ws74"
server_key: "7873f68716b2f36f58b98a1db8a39ebd"
server_key_iv: "323abda9a24fc3e2deeb7c7228f4da08"
server_key_version: "encryptionkey@0"
I20260926 06:56:51.947111 21962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "59096d362b4e4ed4bf52ea7286c8575c"
format_stamp: "Formatted at 2026-09-26 06:56:51 on dist-test-slave-ws74"
server_key: "7873f68716b2f36f58b98a1db8a39ebd"
server_key_iv: "323abda9a24fc3e2deeb7c7228f4da08"
server_key_version: "encryptionkey@0"
I20260926 06:56:51.948421 21962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:56:51.949083 21977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:51.949357 21962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:51.949501 21962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "59096d362b4e4ed4bf52ea7286c8575c"
format_stamp: "Formatted at 2026-09-26 06:56:51 on dist-test-slave-ws74"
server_key: "7873f68716b2f36f58b98a1db8a39ebd"
server_key_iv: "323abda9a24fc3e2deeb7c7228f4da08"
server_key_version: "encryptionkey@0"
I20260926 06:56:51.949579 21962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:51.966504 21962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:51.968199 21962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:51.968358 21962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:51.972280 21962 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43539
I20260926 06:56:51.972316 22029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43539 every 8 connection(s)
I20260926 06:56:51.972675 21962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:51.973623 22030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:51.975719 22030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c: Bootstrap starting.
I20260926 06:56:51.976218 22030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:51.976470 22030 log.cc:824] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:51.977113 22030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c: No bootstrap required, opened a new log
I20260926 06:56:51.978622 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 21962
I20260926 06:56:51.978741 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:51.978919 767 external_mini_cluster.cc:1428] Setting key 5259dcad3c98d9457293a0379289b497
I20260926 06:56:51.978929 22030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59096d362b4e4ed4bf52ea7286c8575c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } }
I20260926 06:56:51.979099 22030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:51.979151 22030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59096d362b4e4ed4bf52ea7286c8575c, State: Initialized, Role: FOLLOWER
I20260926 06:56:51.979293 22030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59096d362b4e4ed4bf52ea7286c8575c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } }
I20260926 06:56:51.979396 22030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:51.979434 22030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:51.979532 22030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:51.980181 22030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59096d362b4e4ed4bf52ea7286c8575c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } }
I20260926 06:56:51.980268 22030 leader_election.cc:302] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59096d362b4e4ed4bf52ea7286c8575c; no voters:
I20260926 06:56:51.980449 22030 leader_election.cc:288] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:51.980481 22035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:51.980629 22035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [term 1 LEADER]: Becoming Leader. State: Replica: 59096d362b4e4ed4bf52ea7286c8575c, State: Running, Role: LEADER
I20260926 06:56:51.980701 22030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:51.980747 22035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59096d362b4e4ed4bf52ea7286c8575c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } }
I20260926 06:56:51.981060 22035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59096d362b4e4ed4bf52ea7286c8575c. Latest consensus state: current_term: 1 leader_uuid: "59096d362b4e4ed4bf52ea7286c8575c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59096d362b4e4ed4bf52ea7286c8575c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } } }
I20260926 06:56:51.981069 22037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59096d362b4e4ed4bf52ea7286c8575c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59096d362b4e4ed4bf52ea7286c8575c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } } }
I20260926 06:56:51.981204 22035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:51.981210 22037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:51.981976 22042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:51.982522 22042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:51.984510 22042 catalog_manager.cc:1347] Generated new cluster ID: 748819a253b9476e86d8cf82d70c008e
I20260926 06:56:51.984563 22042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:56:51 dist-test-slave-ws74 krb5kdc[21949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 06:56:51.991150 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:51.979377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:52107 (local address 127.0.0.1:43539)
0926 06:56:51.979561 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:51.979570 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:51.979782 (+ 212us) server_negotiation.cc:407] Connection header received
0926 06:56:51.980114 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:51.980139 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:51.980185 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:51.980332 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:51.981204 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:51.982745 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:51.983673 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:51.984223 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:51.984344 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:51.987215 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:51.987219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:51.987220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:51.987231 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:51.989313 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:51.989841 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:51.989845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:51.989845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:51.989965 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:51.990240 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:51.990241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:51.990241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:51.990560 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:51.990702 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:51.990866 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 06:56:51.990967 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260926 06:56:51.992187 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:51.995203 22042 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:51.995893 22042 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:52.000674 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:51.993683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:46497 (local address 127.0.0.1:43539)
0926 06:56:51.993720 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:51.993722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:51.993868 (+ 146us) server_negotiation.cc:407] Connection header received
0926 06:56:51.993954 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:51.993955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:51.993963 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:51.993990 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:51.994637 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:51.995320 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:51.996305 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:51.996632 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:51.996668 (+ 36us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:51.998039 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:51.998040 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:51.998040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:51.998046 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:51.999521 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:51.999892 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:51.999893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:51.999893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:51.999940 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:52.000139 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:52.000139 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:52.000140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:52.000244 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:52.000320 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:52.000531 (+ 211us) server_negotiation.cc:299] Negotiation successful
0926 06:56:52.000590 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:56:52.002898 22042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59096d362b4e4ed4bf52ea7286c8575c: Generated new TSK 0
I20260926 06:56:52.003065 22042 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:52.005232 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 21962
W20260926 06:56:52.009684 21947 connection.cc:535] client connection to 127.0.0.1:43539 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43539): Connection reset by peer (error 104)
2026-09-26T06:56:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:52 dist-test-slave-ws74 krb5kdc[22061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:52 dist-test-slave-ws74 krb5kdc[22061](info): set up 2 sockets
Sep 26 06:56:52 dist-test-slave-ws74 krb5kdc[22061](info): commencing operation
krb5kdc: starting...
W20260926 06:56:53.145624 767 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:53 dist-test-slave-ws74 krb5kdc[22061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:56:53.266294 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:53.348198 22074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:53.348440 22074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:53.348480 22074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:53.349736 22074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:53.349795 22074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:53.349820 22074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:53.349851 22074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:53.349876 22074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:53.351334 22074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35369
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:53.351604 22074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:53.351822 22074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:56:53.356673 22074 server_base.cc:1034] running on GCE node
W20260926 06:56:53.356642 22080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:53.356769 22082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:53.356654 22079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:53.357352 22074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:53.357749 22074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:53.358937 22074 hybrid_clock.cc:648] HybridClock initialized: now 1790405813358921 us; error 30 us; skew 500 ppm
Sep 26 06:56:53 dist-test-slave-ws74 krb5kdc[22061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:53.361928 22074 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:56:53.362412 22074 webserver.cc:460] Webserver started at http://127.0.0.1:37367/ using document root <none> and password file <none>
I20260926 06:56:53.362679 22074 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:53.362730 22074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:53.362879 22074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:53.364060 22074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "c6728afc3a9e43e98d2af3ebd88abb8e"
format_stamp: "Formatted at 2026-09-26 06:56:53 on dist-test-slave-ws74"
server_key: "68cf39762e0a1f12c183b219d19d3a0c"
server_key_iv: "cf0f4bb92396a354662a2d890eb28896"
server_key_version: "encryptionkey@0"
I20260926 06:56:53.364452 22074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "c6728afc3a9e43e98d2af3ebd88abb8e"
format_stamp: "Formatted at 2026-09-26 06:56:53 on dist-test-slave-ws74"
server_key: "68cf39762e0a1f12c183b219d19d3a0c"
server_key_iv: "cf0f4bb92396a354662a2d890eb28896"
server_key_version: "encryptionkey@0"
I20260926 06:56:53.365949 22074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:56:53.366737 22089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:53.366972 22074 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:56:53.367090 22074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "c6728afc3a9e43e98d2af3ebd88abb8e"
format_stamp: "Formatted at 2026-09-26 06:56:53 on dist-test-slave-ws74"
server_key: "68cf39762e0a1f12c183b219d19d3a0c"
server_key_iv: "cf0f4bb92396a354662a2d890eb28896"
server_key_version: "encryptionkey@0"
I20260926 06:56:53.367165 22074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:53.377175 22074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:53.379101 22074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:53.379274 22074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:53.383544 22074 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35369
I20260926 06:56:53.383574 22141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35369 every 8 connection(s)
I20260926 06:56:53.383958 22074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:53.384918 22142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:53.387140 22142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e: Bootstrap starting.
I20260926 06:56:53.387712 22142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:53.388000 22142 log.cc:824] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:53.388697 22142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e: No bootstrap required, opened a new log
I20260926 06:56:53.390425 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 22074
I20260926 06:56:53.390571 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:53.390471 22142 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6728afc3a9e43e98d2af3ebd88abb8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } }
I20260926 06:56:53.390671 22142 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:53.390722 22142 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6728afc3a9e43e98d2af3ebd88abb8e, State: Initialized, Role: FOLLOWER
I20260926 06:56:53.390800 767 external_mini_cluster.cc:1428] Setting key 42e5135c04203538eba99833fbb71026
I20260926 06:56:53.390868 22142 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6728afc3a9e43e98d2af3ebd88abb8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } }
I20260926 06:56:53.390939 22142 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:53.391016 22142 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:53.391072 22142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:53.392148 22142 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6728afc3a9e43e98d2af3ebd88abb8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } }
I20260926 06:56:53.392267 22142 leader_election.cc:302] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6728afc3a9e43e98d2af3ebd88abb8e; no voters:
I20260926 06:56:53.392422 22142 leader_election.cc:288] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:53.392457 22147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:53.392566 22147 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [term 1 LEADER]: Becoming Leader. State: Replica: c6728afc3a9e43e98d2af3ebd88abb8e, State: Running, Role: LEADER
I20260926 06:56:53.392696 22142 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:53.392700 22147 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6728afc3a9e43e98d2af3ebd88abb8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } }
I20260926 06:56:53.393162 22149 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6728afc3a9e43e98d2af3ebd88abb8e. Latest consensus state: current_term: 1 leader_uuid: "c6728afc3a9e43e98d2af3ebd88abb8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6728afc3a9e43e98d2af3ebd88abb8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } } }
I20260926 06:56:53.393265 22149 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:53.393463 22148 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6728afc3a9e43e98d2af3ebd88abb8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6728afc3a9e43e98d2af3ebd88abb8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } } }
I20260926 06:56:53.393564 22148 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e [sys.catalog]: This master's current role is: LEADER
W20260926 06:56:53.394569 22163 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:56:53.394657 22163 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:56:53.394732 22156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:53.394861 22156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:53.396318 22156 catalog_manager.cc:1347] Generated new cluster ID: 733a991c62734ebdbf7eea811810382d
I20260926 06:56:53.396374 22156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:53.398597 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:53.391298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:48785 (local address 127.0.0.1:35369)
0926 06:56:53.391496 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:53.391505 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:53.391539 (+ 34us) server_negotiation.cc:407] Connection header received
0926 06:56:53.391636 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:53.391641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:53.391685 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:53.391795 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:53.392026 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:53.393312 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:53.394521 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:53.395012 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:53.395131 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:53.397960 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:53.397964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:53.397964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:53.398153 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:53.398184 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:53.398195 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:53.398292 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:53.398380 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 06:56:53.398409 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
I20260926 06:56:53.399684 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:53.405277 22156 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:53.405982 22156 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:53.408052 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:53.401030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:55717 (local address 127.0.0.1:35369)
0926 06:56:53.401071 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:53.401073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:53.404933 (+ 3860us) server_negotiation.cc:407] Connection header received
0926 06:56:53.405047 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:53.405049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:53.405061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:53.405094 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:53.405340 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:53.406045 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:53.406782 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:53.407016 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:53.407104 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:53.407783 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:53.407784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:53.407784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:53.407788 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:53.407802 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:53.407807 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:53.407860 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:53.407966 (+ 106us) server_negotiation.cc:299] Negotiation successful
0926 06:56:53.407978 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:56:53.412525 22156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6728afc3a9e43e98d2af3ebd88abb8e: Generated new TSK 0
I20260926 06:56:53.412691 22156 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:53.416072 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 22074
W20260926 06:56:53.419921 22059 connection.cc:535] client connection to 127.0.0.1:35369 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35369): Connection reset by peer (error 104)
2026-09-26T06:56:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:53 dist-test-slave-ws74 krb5kdc[22173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:53 dist-test-slave-ws74 krb5kdc[22173](info): set up 2 sockets
Sep 26 06:56:53 dist-test-slave-ws74 krb5kdc[22173](info): commencing operation
krb5kdc: starting...
W20260926 06:56:54.453792 767 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:54 dist-test-slave-ws74 krb5kdc[22173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:56:54.565616 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37357
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:54.644640 22186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:54.644851 22186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:54.644874 22186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:54.646144 22186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:54.646190 22186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:54.646205 22186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:54.646217 22186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:54.646230 22186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:54.647707 22186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37357
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:54.647957 22186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:54.648188 22186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:54.652915 22191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:54.652943 22194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:54.653095 22186 server_base.cc:1034] running on GCE node
W20260926 06:56:54.653139 22192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:54.653409 22186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:54.653748 22186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:54.654896 22186 hybrid_clock.cc:648] HybridClock initialized: now 1790405814654874 us; error 39 us; skew 500 ppm
Sep 26 06:56:54 dist-test-slave-ws74 krb5kdc[22173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405814, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:54.662266 22186 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:56:54.662750 22186 webserver.cc:460] Webserver started at http://127.0.0.1:45653/ using document root <none> and password file <none>
I20260926 06:56:54.662978 22186 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:54.663024 22186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:54.663189 22186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:54.664243 22186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "f22f4544bb3a41558ba054078ce08cb9"
format_stamp: "Formatted at 2026-09-26 06:56:54 on dist-test-slave-ws74"
server_key: "f90ef5795ff74f44615b83b5e8ffd49a"
server_key_iv: "18521dff182ae5d40a63c22b10766318"
server_key_version: "encryptionkey@0"
I20260926 06:56:54.664608 22186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "f22f4544bb3a41558ba054078ce08cb9"
format_stamp: "Formatted at 2026-09-26 06:56:54 on dist-test-slave-ws74"
server_key: "f90ef5795ff74f44615b83b5e8ffd49a"
server_key_iv: "18521dff182ae5d40a63c22b10766318"
server_key_version: "encryptionkey@0"
I20260926 06:56:54.666035 22186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:56:54.666775 22201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:54.666949 22186 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:56:54.667048 22186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "f22f4544bb3a41558ba054078ce08cb9"
format_stamp: "Formatted at 2026-09-26 06:56:54 on dist-test-slave-ws74"
server_key: "f90ef5795ff74f44615b83b5e8ffd49a"
server_key_iv: "18521dff182ae5d40a63c22b10766318"
server_key_version: "encryptionkey@0"
I20260926 06:56:54.667128 22186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:54.670425 22186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:54.672317 22186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:54.672470 22186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:54.676486 22186 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37357
I20260926 06:56:54.676514 22253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37357 every 8 connection(s)
I20260926 06:56:54.676820 22186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:54.677772 22254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:54.679838 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 22186
I20260926 06:56:54.679955 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:54.680014 22254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9: Bootstrap starting.
I20260926 06:56:54.680146 767 external_mini_cluster.cc:1428] Setting key d324df5375dd656e4b71a99fc2d5feb0
I20260926 06:56:54.680728 22254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:54.681030 22254 log.cc:824] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:54.681806 22254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9: No bootstrap required, opened a new log
I20260926 06:56:54.682729 22257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:54.680605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:54775 (local address 127.0.0.1:37357)
0926 06:56:54.680799 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:54.680808 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:54.680839 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:56:54.681421 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:54.681429 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:54.681465 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:54.681599 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:54.682280 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:54.682284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:54.682284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:54.682288 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:54.682337 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:54.682347 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:54.682437 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:54.682477 (+ 40us) server_negotiation.cc:299] Negotiation successful
0926 06:56:54.682538 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260926 06:56:54.683666 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:54.684466 22254 raft_consensus.cc:348] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22f4544bb3a41558ba054078ce08cb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37357 } }
I20260926 06:56:54.684645 22254 raft_consensus.cc:374] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:54.684677 22254 raft_consensus.cc:729] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22f4544bb3a41558ba054078ce08cb9, State: Initialized, Role: FOLLOWER
I20260926 06:56:54.684803 22254 consensus_queue.cc:260] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22f4544bb3a41558ba054078ce08cb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37357 } }
I20260926 06:56:54.684908 22254 raft_consensus.cc:388] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:54.684955 22254 raft_consensus.cc:482] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:54.685010 22254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:54.685715 22254 raft_consensus.cc:504] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22f4544bb3a41558ba054078ce08cb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37357 } }
I20260926 06:56:54.685824 22254 leader_election.cc:302] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f22f4544bb3a41558ba054078ce08cb9; no voters:
I20260926 06:56:54.685984 22254 leader_election.cc:288] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:54.686050 22264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:54.686200 22264 raft_consensus.cc:686] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [term 1 LEADER]: Becoming Leader. State: Replica: f22f4544bb3a41558ba054078ce08cb9, State: Running, Role: LEADER
I20260926 06:56:54.686237 22254 sys_catalog.cc:564] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:54.686321 22257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:54.685252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:45999 (local address 127.0.0.1:37357)
0926 06:56:54.685284 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:54.685286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:54.685362 (+ 76us) server_negotiation.cc:407] Connection header received
0926 06:56:54.685444 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:54.685445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:54.685449 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:54.685513 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:54.686050 (+ 537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:54.686052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:54.686052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:54.686057 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:54.686071 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:54.686075 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:54.686106 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:54.686215 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 06:56:54.686227 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:56:54.686318 22264 consensus_queue.cc:237] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22f4544bb3a41558ba054078ce08cb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37357 } }
I20260926 06:56:54.686691 22264 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f22f4544bb3a41558ba054078ce08cb9. Latest consensus state: current_term: 1 leader_uuid: "f22f4544bb3a41558ba054078ce08cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22f4544bb3a41558ba054078ce08cb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37357 } } }
I20260926 06:56:54.686772 22264 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:54.686707 22265 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f22f4544bb3a41558ba054078ce08cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22f4544bb3a41558ba054078ce08cb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37357 } } }
I20260926 06:56:54.687023 22265 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:54.687134 22270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:54.687772 22270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:54.689246 22270 catalog_manager.cc:1347] Generated new cluster ID: f5fb3a3d7e544dbaa6e3efb2ac27a961
I20260926 06:56:54.689301 22270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:54.730520 22270 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:54.730700 22270 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:54.740216 22270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f22f4544bb3a41558ba054078ce08cb9: Generated new TSK 0
I20260926 06:56:54.740414 22270 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:54.742063 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 22186
2026-09-26T06:56:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1326 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:54 dist-test-slave-ws74 krb5kdc[22285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:54 dist-test-slave-ws74 krb5kdc[22285](info): set up 2 sockets
Sep 26 06:56:54 dist-test-slave-ws74 krb5kdc[22285](info): commencing operation
krb5kdc: starting...
W20260926 06:56:55.773432 767 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:55 dist-test-slave-ws74 krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:56:55.888628 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:39859
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:39859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:55.967410 22298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:55.967595 22298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:55.967617 22298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:55.968662 22298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:55.968693 22298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:55.968707 22298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:55.968719 22298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:55.968731 22298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:55.970096 22298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:39859
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:39859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:55.970265 22298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:55.970420 22298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:55.974910 22303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:55.975073 22304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:55.974978 22298 server_base.cc:1034] running on GCE node
W20260926 06:56:55.974924 22306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:55.975374 22298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:55.975641 22298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:55.976790 22298 hybrid_clock.cc:648] HybridClock initialized: now 1790405815976781 us; error 33 us; skew 500 ppm
Sep 26 06:56:55 dist-test-slave-ws74 krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405815, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:55.979164 22298 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:56:55.979635 22298 webserver.cc:460] Webserver started at http://127.0.191.254:39981/ using document root <none> and password file <none>
I20260926 06:56:55.979846 22298 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:55.979889 22298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:55.980043 22298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:55.981076 22298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "c19e897b2b9546cc8f77e1493a575ec2"
format_stamp: "Formatted at 2026-09-26 06:56:55 on dist-test-slave-ws74"
server_key: "500efe6b0ec0fc066b6a8826e3ed5103"
server_key_iv: "18cbc7ea7b00dd3edd8a350588069b78"
server_key_version: "encryptionkey@0"
I20260926 06:56:55.981449 22298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "c19e897b2b9546cc8f77e1493a575ec2"
format_stamp: "Formatted at 2026-09-26 06:56:55 on dist-test-slave-ws74"
server_key: "500efe6b0ec0fc066b6a8826e3ed5103"
server_key_iv: "18cbc7ea7b00dd3edd8a350588069b78"
server_key_version: "encryptionkey@0"
I20260926 06:56:55.982712 22298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:56:55.983408 22313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:55.983577 22298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:56:55.983661 22298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "c19e897b2b9546cc8f77e1493a575ec2"
format_stamp: "Formatted at 2026-09-26 06:56:55 on dist-test-slave-ws74"
server_key: "500efe6b0ec0fc066b6a8826e3ed5103"
server_key_iv: "18cbc7ea7b00dd3edd8a350588069b78"
server_key_version: "encryptionkey@0"
I20260926 06:56:55.983712 22298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:56.001499 22298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:56.003633 22298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:56.003737 22298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:56.007265 22298 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:39859
I20260926 06:56:56.007315 22365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:39859 every 8 connection(s)
I20260926 06:56:56.007622 22298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:56.008453 22366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:56.010659 22366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2: Bootstrap starting.
I20260926 06:56:56.011219 22366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:56.011494 22366 log.cc:824] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:56.012166 22366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2: No bootstrap required, opened a new log
I20260926 06:56:56.013046 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 22298
I20260926 06:56:56.013166 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:56.013361 767 external_mini_cluster.cc:1428] Setting key 7a24d44124ead62c4140a20cc9c77b29
I20260926 06:56:56.014249 22366 raft_consensus.cc:348] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c19e897b2b9546cc8f77e1493a575ec2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39859 } }
I20260926 06:56:56.014420 22366 raft_consensus.cc:374] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:56.014461 22366 raft_consensus.cc:729] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c19e897b2b9546cc8f77e1493a575ec2, State: Initialized, Role: FOLLOWER
I20260926 06:56:56.014622 22366 consensus_queue.cc:260] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c19e897b2b9546cc8f77e1493a575ec2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39859 } }
I20260926 06:56:56.014712 22366 raft_consensus.cc:388] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:56.014747 22366 raft_consensus.cc:482] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:56.014799 22366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:56.015743 22366 raft_consensus.cc:504] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c19e897b2b9546cc8f77e1493a575ec2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39859 } }
I20260926 06:56:56.015861 22366 leader_election.cc:302] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c19e897b2b9546cc8f77e1493a575ec2; no voters:
I20260926 06:56:56.016048 22366 leader_election.cc:288] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:56.016067 22371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:56.016170 22371 raft_consensus.cc:686] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [term 1 LEADER]: Becoming Leader. State: Replica: c19e897b2b9546cc8f77e1493a575ec2, State: Running, Role: LEADER
I20260926 06:56:56.016249 22371 consensus_queue.cc:237] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c19e897b2b9546cc8f77e1493a575ec2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39859 } }
I20260926 06:56:56.016291 22366 sys_catalog.cc:564] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:56.016651 22373 sys_catalog.cc:455] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c19e897b2b9546cc8f77e1493a575ec2. Latest consensus state: current_term: 1 leader_uuid: "c19e897b2b9546cc8f77e1493a575ec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c19e897b2b9546cc8f77e1493a575ec2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39859 } } }
I20260926 06:56:56.016718 22373 sys_catalog.cc:458] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:56.016896 22375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:56.016876 22372 sys_catalog.cc:455] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c19e897b2b9546cc8f77e1493a575ec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c19e897b2b9546cc8f77e1493a575ec2" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39859 } } }
I20260926 06:56:56.016938 22372 sys_catalog.cc:458] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2 [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:56.017364 22375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:56.018929 22375 catalog_manager.cc:1347] Generated new cluster ID: ae92f25a76ba4328a3405ba6cb6e6d3b
I20260926 06:56:56.018986 22375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:56:56 dist-test-slave-ws74 krb5kdc[22285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:56.024288 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:56.013906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39735 (local address 127.0.191.254:39859)
0926 06:56:56.014072 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:56.014079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:56.014102 (+ 23us) server_negotiation.cc:407] Connection header received
0926 06:56:56.014212 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:56.014217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:56.014258 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:56.014366 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:56.015188 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:56.016191 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:56.017360 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:56.018096 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:56.018202 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:56.020655 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:56.020659 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:56.020659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:56.020670 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:56.022421 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:56.022818 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:56.022821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:56.022822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:56.022935 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:56.023176 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:56.023177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:56.023178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:56.023419 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:56.024017 (+ 598us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:56.024083 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 06:56:56.024157 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260926 06:56:56.025158 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:56.035691 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:56.029694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49295 (local address 127.0.191.254:39859)
0926 06:56:56.029873 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:56.029875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:56.029880 (+ 5us) server_negotiation.cc:407] Connection header received
0926 06:56:56.029895 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:56.029895 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:56.029902 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:56.029960 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:56.030467 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:56.030959 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:56.031654 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:56.031879 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:56.031964 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:56.033100 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:56.033101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:56.033102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:56.033108 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:56.034529 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:56.034886 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:56.034887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:56.034887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:56.034938 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:56.035171 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:56.035172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:56.035173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:56.035279 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:56.035382 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:56.035506 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 06:56:56.035573 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260926 06:56:56.046795 22375 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:56.047484 22375 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:56.054627 22375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c19e897b2b9546cc8f77e1493a575ec2: Generated new TSK 0
I20260926 06:56:56.054883 22375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:56.062815 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 22298
W20260926 06:56:56.067375 22283 connection.cc:535] client connection to 127.0.191.254:39859 recv error: Network error: failed to read from TLS socket (remote: 127.0.191.254:39859): Connection reset by peer (error 104)
2026-09-26T06:56:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:56 dist-test-slave-ws74 krb5kdc[22397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:56 dist-test-slave-ws74 krb5kdc[22397](info): set up 2 sockets
Sep 26 06:56:56 dist-test-slave-ws74 krb5kdc[22397](info): commencing operation
krb5kdc: starting...
W20260926 06:56:57.101949 767 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:57 dist-test-slave-ws74 krb5kdc[22397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:56:57.215641 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:40605
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:40605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:57.293151 22410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:57.293381 22410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:57.293440 22410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:57.294553 22410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:57.294623 22410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:57.294647 22410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:57.294677 22410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:57.294703 22410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:57.296216 22410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:40605
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:40605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:57.296463 22410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:57.296693 22410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:57.301434 22415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:57.301440 22416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:57.301514 22418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:57.301789 22410 server_base.cc:1034] running on GCE node
I20260926 06:56:57.302000 22410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:57.302232 22410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:57.303366 22410 hybrid_clock.cc:648] HybridClock initialized: now 1790405817303350 us; error 33 us; skew 500 ppm
Sep 26 06:56:57 dist-test-slave-ws74 krb5kdc[22397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405817, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:57.305823 22410 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:56:57.306252 22410 webserver.cc:460] Webserver started at http://127.0.191.254:40783/ using document root <none> and password file <none>
I20260926 06:56:57.306435 22410 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:57.306473 22410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:57.306559 22410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:57.307442 22410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "a1d379f9dde847b48a8520c8f761057b"
format_stamp: "Formatted at 2026-09-26 06:56:57 on dist-test-slave-ws74"
server_key: "97d256710410ad3b8ec7c471a59cad64"
server_key_iv: "8c90dbf702a56b88d0cac51b66b82bdb"
server_key_version: "encryptionkey@0"
I20260926 06:56:57.307730 22410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "a1d379f9dde847b48a8520c8f761057b"
format_stamp: "Formatted at 2026-09-26 06:56:57 on dist-test-slave-ws74"
server_key: "97d256710410ad3b8ec7c471a59cad64"
server_key_iv: "8c90dbf702a56b88d0cac51b66b82bdb"
server_key_version: "encryptionkey@0"
I20260926 06:56:57.308905 22410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:56:57.309563 22425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:57.309733 22410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:57.309823 22410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "a1d379f9dde847b48a8520c8f761057b"
format_stamp: "Formatted at 2026-09-26 06:56:57 on dist-test-slave-ws74"
server_key: "97d256710410ad3b8ec7c471a59cad64"
server_key_iv: "8c90dbf702a56b88d0cac51b66b82bdb"
server_key_version: "encryptionkey@0"
I20260926 06:56:57.309902 22410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:57.328529 22410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:57.330886 22410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:57.331049 22410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:57.335160 22410 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:40605
I20260926 06:56:57.335193 22477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:40605 every 8 connection(s)
I20260926 06:56:57.335547 22410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:57.336349 22478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:57.338541 22478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b: Bootstrap starting.
I20260926 06:56:57.339039 22478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:57.339264 22478 log.cc:824] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:57.339648 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 22410
I20260926 06:56:57.339779 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:57.339985 767 external_mini_cluster.cc:1428] Setting key bdf87c5b2e3a8711a4edee5b8fb6874e
I20260926 06:56:57.339995 22478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b: No bootstrap required, opened a new log
I20260926 06:56:57.341849 22478 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d379f9dde847b48a8520c8f761057b" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40605 } }
I20260926 06:56:57.342013 22478 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:57.342068 22478 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d379f9dde847b48a8520c8f761057b, State: Initialized, Role: FOLLOWER
I20260926 06:56:57.342226 22478 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d379f9dde847b48a8520c8f761057b" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40605 } }
I20260926 06:56:57.342311 22478 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:57.342345 22478 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:57.342394 22478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:57.343083 22478 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d379f9dde847b48a8520c8f761057b" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40605 } }
I20260926 06:56:57.343189 22478 leader_election.cc:302] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1d379f9dde847b48a8520c8f761057b; no voters:
I20260926 06:56:57.343350 22478 leader_election.cc:288] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:57.343430 22483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:57.343595 22483 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [term 1 LEADER]: Becoming Leader. State: Replica: a1d379f9dde847b48a8520c8f761057b, State: Running, Role: LEADER
I20260926 06:56:57.343600 22478 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:57.343740 22483 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d379f9dde847b48a8520c8f761057b" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40605 } }
I20260926 06:56:57.344110 22485 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1d379f9dde847b48a8520c8f761057b. Latest consensus state: current_term: 1 leader_uuid: "a1d379f9dde847b48a8520c8f761057b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d379f9dde847b48a8520c8f761057b" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40605 } } }
I20260926 06:56:57.344198 22485 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:57.344424 22484 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1d379f9dde847b48a8520c8f761057b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d379f9dde847b48a8520c8f761057b" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40605 } } }
I20260926 06:56:57.344543 22484 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:57.344897 22490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:57.345423 22490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:57.347328 22482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:57.340454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35771 (local address 127.0.191.254:40605)
0926 06:56:57.340625 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:57.340632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:57.340658 (+ 26us) server_negotiation.cc:407] Connection header received
0926 06:56:57.341835 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:57.341840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:57.341884 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:57.341990 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:57.342274 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:57.344010 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:57.345189 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:57.345824 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:57.345935 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:57.346629 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:57.346632 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:57.346633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:57.346842 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:57.346891 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:57.346906 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:57.347009 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:57.347090 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 06:56:57.347117 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260926 06:56:57.347352 22490 catalog_manager.cc:1347] Generated new cluster ID: 5b0170b5d3cc4f6d8dd2d05a5ed44a21
I20260926 06:56:57.347393 22490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:57.348202 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:57.356324 22482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:57.349510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51789 (local address 127.0.191.254:40605)
0926 06:56:57.353463 (+ 3953us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:57.353465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:57.353607 (+ 142us) server_negotiation.cc:407] Connection header received
0926 06:56:57.353681 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:57.353682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:57.353693 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:57.353737 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:57.353908 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:57.354392 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:57.355108 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:57.355334 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:57.355449 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:57.356084 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:57.356086 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:57.356086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:57.356090 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:57.356103 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:57.356107 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:57.356159 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:57.356219 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 06:56:57.356229 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3932}
I20260926 06:56:57.361615 22490 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:57.362100 22490 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:57.366460 22490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1d379f9dde847b48a8520c8f761057b: Generated new TSK 0
I20260926 06:56:57.366616 22490 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:57.368695 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 22410
W20260926 06:56:57.373095 22395 connection.cc:535] client connection to 127.0.191.254:40605 recv error: Network error: failed to read from TLS socket (remote: 127.0.191.254:40605): Connection reset by peer (error 104)
2026-09-26T06:56:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:57 dist-test-slave-ws74 krb5kdc[22509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:57 dist-test-slave-ws74 krb5kdc[22509](info): set up 2 sockets
Sep 26 06:56:57 dist-test-slave-ws74 krb5kdc[22509](info): commencing operation
krb5kdc: starting...
W20260926 06:56:58.389760 767 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:58 dist-test-slave-ws74 krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:56:58.520246 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:44607
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:44267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:44607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:58.599267 22522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:58.599505 22522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:58.599545 22522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:58.600693 22522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:58.600761 22522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:58.600785 22522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:58.600814 22522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:58.600841 22522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:58.602308 22522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:44267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:44607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:44607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:58.602581 22522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:58.602852 22522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:58.607631 22528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:58.607673 22530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:58.607646 22527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:58.607875 22522 server_base.cc:1034] running on GCE node
I20260926 06:56:58.608098 22522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:58.608332 22522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:58.609516 22522 hybrid_clock.cc:648] HybridClock initialized: now 1790405818609502 us; error 27 us; skew 500 ppm
Sep 26 06:56:58 dist-test-slave-ws74 krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405818, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:58.612038 22522 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:56:58.612514 22522 webserver.cc:460] Webserver started at http://127.0.191.254:41323/ using document root <none> and password file <none>
I20260926 06:56:58.612707 22522 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:58.612747 22522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:58.612833 22522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:58.613893 22522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "f295d5820b604a11b5e014701eda643b"
format_stamp: "Formatted at 2026-09-26 06:56:58 on dist-test-slave-ws74"
server_key: "ced8312f04a54b93a666e2dac2e59e5b"
server_key_iv: "0bc15b15667306073f1048488520bbb7"
server_key_version: "encryptionkey@0"
I20260926 06:56:58.614183 22522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "f295d5820b604a11b5e014701eda643b"
format_stamp: "Formatted at 2026-09-26 06:56:58 on dist-test-slave-ws74"
server_key: "ced8312f04a54b93a666e2dac2e59e5b"
server_key_iv: "0bc15b15667306073f1048488520bbb7"
server_key_version: "encryptionkey@0"
I20260926 06:56:58.615461 22522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:56:58.616173 22537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:58.616374 22522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:56:58.616469 22522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "f295d5820b604a11b5e014701eda643b"
format_stamp: "Formatted at 2026-09-26 06:56:58 on dist-test-slave-ws74"
server_key: "ced8312f04a54b93a666e2dac2e59e5b"
server_key_iv: "0bc15b15667306073f1048488520bbb7"
server_key_version: "encryptionkey@0"
I20260926 06:56:58.616544 22522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:58.619763 22522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:58.621697 22522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:58.621845 22522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:58.625504 22522 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:44607
I20260926 06:56:58.625556 22589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:44607 every 8 connection(s)
I20260926 06:56:58.625841 22522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:58.626813 22590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:58.628890 22590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b: Bootstrap starting.
I20260926 06:56:58.629439 22590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:58.629686 22590 log.cc:824] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:58.630297 22590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b: No bootstrap required, opened a new log
I20260926 06:56:58.631785 22590 raft_consensus.cc:348] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f295d5820b604a11b5e014701eda643b" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44607 } }
I20260926 06:56:58.631915 22590 raft_consensus.cc:374] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:58.631937 22590 raft_consensus.cc:729] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f295d5820b604a11b5e014701eda643b, State: Initialized, Role: FOLLOWER
I20260926 06:56:58.632043 22590 consensus_queue.cc:260] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f295d5820b604a11b5e014701eda643b" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44607 } }
I20260926 06:56:58.632122 22590 raft_consensus.cc:388] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:58.632158 22590 raft_consensus.cc:482] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:58.632234 22590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:58.632757 22590 raft_consensus.cc:504] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f295d5820b604a11b5e014701eda643b" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44607 } }
I20260926 06:56:58.632874 22590 leader_election.cc:302] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f295d5820b604a11b5e014701eda643b; no voters:
I20260926 06:56:58.633049 22590 leader_election.cc:288] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:58.633091 22593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:58.633303 22590 sys_catalog.cc:564] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:58.633303 22593 raft_consensus.cc:686] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [term 1 LEADER]: Becoming Leader. State: Replica: f295d5820b604a11b5e014701eda643b, State: Running, Role: LEADER
I20260926 06:56:58.633453 22593 consensus_queue.cc:237] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f295d5820b604a11b5e014701eda643b" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44607 } }
I20260926 06:56:58.633823 22595 sys_catalog.cc:455] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f295d5820b604a11b5e014701eda643b. Latest consensus state: current_term: 1 leader_uuid: "f295d5820b604a11b5e014701eda643b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f295d5820b604a11b5e014701eda643b" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44607 } } }
I20260926 06:56:58.633965 22595 sys_catalog.cc:458] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:58.633948 22594 sys_catalog.cc:455] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f295d5820b604a11b5e014701eda643b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f295d5820b604a11b5e014701eda643b" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44607 } } }
I20260926 06:56:58.634202 22602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:58.634220 22594 sys_catalog.cc:458] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:58.634738 22602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:56:58.634758 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 22522
I20260926 06:56:58.634862 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:58.635061 767 external_mini_cluster.cc:1428] Setting key e4f21b052e8f61b98c4cc8f0e8cfb471
I20260926 06:56:58.637181 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:58.635502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41739 (local address 127.0.191.254:44607)
0926 06:56:58.635667 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:58.635676 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:58.635735 (+ 59us) server_negotiation.cc:407] Connection header received
0926 06:56:58.635851 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:58.635858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:58.635898 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:58.636018 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:58.636746 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:58.636750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:58.636750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:58.636755 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:58.636811 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:58.636827 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:58.636904 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:58.636954 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 06:56:58.636993 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":85,"threads_started":1}
I20260926 06:56:58.638149 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:58.640623 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:58.639618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36013 (local address 127.0.191.254:44607)
0926 06:56:58.639656 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:58.639658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:58.639793 (+ 135us) server_negotiation.cc:407] Connection header received
0926 06:56:58.639852 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:58.639853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:58.639857 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:58.639901 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:58.640398 (+ 497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:58.640399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:58.640399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:56:58.640402 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:58.640414 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:56:58.640418 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:58.640456 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:58.640515 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 06:56:58.640532 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:56:58.641780 22602 catalog_manager.cc:1347] Generated new cluster ID: d332775845894f75b031d4bf33b384f0
I20260926 06:56:58.641855 22602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:58.665925 22602 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:58.666131 22602 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:58.670383 22602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f295d5820b604a11b5e014701eda643b: Generated new TSK 0
I20260926 06:56:58.670573 22602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:58.678396 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 22522
2026-09-26T06:56:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1309 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:56:58 dist-test-slave-ws74 krb5kdc[22621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:56:58 dist-test-slave-ws74 krb5kdc[22621](info): set up 2 sockets
Sep 26 06:56:58 dist-test-slave-ws74 krb5kdc[22621](info): commencing operation
krb5kdc: starting...
W20260926 06:56:59.697440 767 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:56:59 dist-test-slave-ws74 krb5kdc[22621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:56:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:56:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:56:59.811506 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:36221
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:36221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:56:59.889570 22634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:56:59.889765 22634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:56:59.889788 22634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:56:59.891055 22634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:56:59.891098 22634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:56:59.891113 22634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:56:59.891126 22634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:56:59.891139 22634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:56:59.892936 22634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:36221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:36221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:56:59.893215 22634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:56:59.893476 22634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:56:59.898283 22639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:56:59.898320 22642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:59.898339 22634 server_base.cc:1034] running on GCE node
W20260926 06:56:59.898319 22640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:56:59.898744 22634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:56:59.899091 22634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:56:59.900345 22634 hybrid_clock.cc:648] HybridClock initialized: now 1790405819900337 us; error 48 us; skew 500 ppm
Sep 26 06:56:59 dist-test-slave-ws74 krb5kdc[22621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:56:59.907294 22634 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:56:59.907742 22634 webserver.cc:460] Webserver started at http://127.0.191.254:34581/ using document root <none> and password file <none>
I20260926 06:56:59.907923 22634 fs_manager.cc:359] Metadata directory not provided
I20260926 06:56:59.907963 22634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:56:59.908051 22634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:56:59.909055 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "aacab1a21f5445a29e0d8e52438752ef"
format_stamp: "Formatted at 2026-09-26 06:56:59 on dist-test-slave-ws74"
server_key: "da2aa22536798c26690c5d72e4b911a1"
server_key_iv: "f083e23380354c73fdcd5132d03bc31b"
server_key_version: "encryptionkey@0"
I20260926 06:56:59.909344 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "aacab1a21f5445a29e0d8e52438752ef"
format_stamp: "Formatted at 2026-09-26 06:56:59 on dist-test-slave-ws74"
server_key: "da2aa22536798c26690c5d72e4b911a1"
server_key_iv: "f083e23380354c73fdcd5132d03bc31b"
server_key_version: "encryptionkey@0"
I20260926 06:56:59.910638 22634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:56:59.911299 22649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:56:59.911461 22634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:56:59.911557 22634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "aacab1a21f5445a29e0d8e52438752ef"
format_stamp: "Formatted at 2026-09-26 06:56:59 on dist-test-slave-ws74"
server_key: "da2aa22536798c26690c5d72e4b911a1"
server_key_iv: "f083e23380354c73fdcd5132d03bc31b"
server_key_version: "encryptionkey@0"
I20260926 06:56:59.911609 22634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:56:59.936002 22634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:56:59.938351 22634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:56:59.938462 22634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:56:59.942201 22634 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:36221
I20260926 06:56:59.942279 22701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:36221 every 8 connection(s)
I20260926 06:56:59.942626 22634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:56:59.943504 22702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:56:59.945720 22702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef: Bootstrap starting.
I20260926 06:56:59.946161 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 22634
I20260926 06:56:59.946280 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:56:59.946447 22702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef: Neither blocks nor log segments found. Creating new log.
I20260926 06:56:59.946480 767 external_mini_cluster.cc:1428] Setting key f000880f1c53a60c43267758ce933b8b
I20260926 06:56:59.946748 22702 log.cc:824] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef: Log is configured to *not* fsync() on all Append() calls
I20260926 06:56:59.947435 22702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef: No bootstrap required, opened a new log
I20260926 06:56:59.949206 22702 raft_consensus.cc:348] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacab1a21f5445a29e0d8e52438752ef" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36221 } }
I20260926 06:56:59.949334 22702 raft_consensus.cc:374] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:56:59.949357 22702 raft_consensus.cc:729] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aacab1a21f5445a29e0d8e52438752ef, State: Initialized, Role: FOLLOWER
I20260926 06:56:59.949530 22702 consensus_queue.cc:260] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacab1a21f5445a29e0d8e52438752ef" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36221 } }
I20260926 06:56:59.949604 22702 raft_consensus.cc:388] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:56:59.949625 22702 raft_consensus.cc:482] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:56:59.949687 22702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:56:59.950199 22702 raft_consensus.cc:504] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacab1a21f5445a29e0d8e52438752ef" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36221 } }
I20260926 06:56:59.950318 22702 leader_election.cc:302] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aacab1a21f5445a29e0d8e52438752ef; no voters:
I20260926 06:56:59.950513 22702 leader_election.cc:288] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:56:59.950541 22707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:56:59.950649 22707 raft_consensus.cc:686] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [term 1 LEADER]: Becoming Leader. State: Replica: aacab1a21f5445a29e0d8e52438752ef, State: Running, Role: LEADER
I20260926 06:56:59.950766 22702 sys_catalog.cc:564] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:56:59.950789 22707 consensus_queue.cc:237] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacab1a21f5445a29e0d8e52438752ef" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36221 } }
I20260926 06:56:59.951107 22707 sys_catalog.cc:455] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader aacab1a21f5445a29e0d8e52438752ef. Latest consensus state: current_term: 1 leader_uuid: "aacab1a21f5445a29e0d8e52438752ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacab1a21f5445a29e0d8e52438752ef" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36221 } } }
I20260926 06:56:59.951242 22707 sys_catalog.cc:458] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:59.951119 22708 sys_catalog.cc:455] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aacab1a21f5445a29e0d8e52438752ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacab1a21f5445a29e0d8e52438752ef" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 36221 } } }
I20260926 06:56:59.951522 22708 sys_catalog.cc:458] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef [sys.catalog]: This master's current role is: LEADER
I20260926 06:56:59.951967 22713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:56:59.952764 22713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 06:56:59 dist-test-slave-ws74 krb5kdc[22621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:56:59.954470 22713 catalog_manager.cc:1347] Generated new cluster ID: f9866217aaea4a1cbca7ef8e010e6c79
I20260926 06:56:59.954522 22713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:56:59.957559 22706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:59.946950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49111 (local address 127.0.191.254:36221)
0926 06:56:59.947142 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:59.947151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:59.947183 (+ 32us) server_negotiation.cc:407] Connection header received
0926 06:56:59.947667 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:59.947673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:59.947720 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:59.947862 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:59.948500 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:59.949426 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:59.950286 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:59.950898 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:59.951295 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:59.953928 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:59.953931 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:59.953931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:59.953940 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:59.955888 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:59.956293 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:59.956295 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:59.956296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:59.956390 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:59.956603 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:59.956604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:59.956604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:59.956872 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:59.957066 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:59.957128 (+ 62us) server_negotiation.cc:299] Negotiation successful
0926 06:56:59.957337 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260926 06:56:59.958429 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:56:59.963975 22713 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:56:59.964716 22713 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:56:59.966776 22713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aacab1a21f5445a29e0d8e52438752ef: Generated new TSK 0
I20260926 06:56:59.966967 22713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:56:59.970302 22706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:56:59.963806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52309 (local address 127.0.191.254:36221)
0926 06:56:59.963842 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:56:59.963844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:56:59.963954 (+ 110us) server_negotiation.cc:407] Connection header received
0926 06:56:59.964046 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:56:59.964047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:56:59.964057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:56:59.964110 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:56:59.964689 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:59.965457 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:59.966425 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:56:59.966748 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:56:59.966816 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:56:59.967905 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:56:59.967907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:56:59.967907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:56:59.967914 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:56:59.969350 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:59.969682 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:59.969683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:59.969683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:59.969740 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:56:59.969876 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:56:59.969877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:56:59.969877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:56:59.969990 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:56:59.970077 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 06:56:59.970136 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 06:56:59.970216 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":20}
I20260926 06:56:59.972098 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 22634
2026-09-26T06:56:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:57:00 dist-test-slave-ws74 krb5kdc[22733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:57:00 dist-test-slave-ws74 krb5kdc[22733](info): set up 2 sockets
Sep 26 06:57:00 dist-test-slave-ws74 krb5kdc[22733](info): commencing operation
krb5kdc: starting...
W20260926 06:57:00.997535 767 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:57:01 dist-test-slave-ws74 krb5kdc[22733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:57:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:57:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:57:01.115972 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:46023
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:46023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:57:01.195026 22746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:57:01.195252 22746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:57:01.195290 22746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:57:01.196415 22746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:57:01.196485 22746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:57:01.196520 22746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:57:01.196550 22746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:57:01.196573 22746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:57:01.198082 22746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:46023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:46023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:57:01.198349 22746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:57:01.198567 22746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:57:01.203245 22746 server_base.cc:1034] running on GCE node
W20260926 06:57:01.203248 22751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:01.203265 22752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:01.203438 22754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:57:01.203717 22746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:57:01.204010 22746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:57:01.205184 22746 hybrid_clock.cc:648] HybridClock initialized: now 1790405821205171 us; error 27 us; skew 500 ppm
Sep 26 06:57:01 dist-test-slave-ws74 krb5kdc[22733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405821, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:57:01.207749 22746 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:57:01.208194 22746 webserver.cc:460] Webserver started at http://127.0.191.254:43865/ using document root <none> and password file <none>
I20260926 06:57:01.208443 22746 fs_manager.cc:359] Metadata directory not provided
I20260926 06:57:01.208488 22746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:57:01.208613 22746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:57:01.209697 22746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "303681c45a1245339842dbabb21ab04e"
format_stamp: "Formatted at 2026-09-26 06:57:01 on dist-test-slave-ws74"
server_key: "66ec6b4c363df741a0d66322ee8e3504"
server_key_iv: "4703b3b3cbd1239a3c346677a7def651"
server_key_version: "encryptionkey@0"
I20260926 06:57:01.210045 22746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "303681c45a1245339842dbabb21ab04e"
format_stamp: "Formatted at 2026-09-26 06:57:01 on dist-test-slave-ws74"
server_key: "66ec6b4c363df741a0d66322ee8e3504"
server_key_iv: "4703b3b3cbd1239a3c346677a7def651"
server_key_version: "encryptionkey@0"
I20260926 06:57:01.211367 22746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:57:01.212067 22761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:57:01.212256 22746 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:57:01.212327 22746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "303681c45a1245339842dbabb21ab04e"
format_stamp: "Formatted at 2026-09-26 06:57:01 on dist-test-slave-ws74"
server_key: "66ec6b4c363df741a0d66322ee8e3504"
server_key_iv: "4703b3b3cbd1239a3c346677a7def651"
server_key_version: "encryptionkey@0"
I20260926 06:57:01.212419 22746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:57:01.228380 22746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:57:01.230657 22746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:57:01.230834 22746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:57:01.234577 22746 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:46023
I20260926 06:57:01.234618 22813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:46023 every 8 connection(s)
I20260926 06:57:01.234943 22746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:57:01.235765 22814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:57:01.238018 22814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e: Bootstrap starting.
I20260926 06:57:01.238593 22814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e: Neither blocks nor log segments found. Creating new log.
I20260926 06:57:01.238873 22814 log.cc:824] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:57:01.239543 22814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e: No bootstrap required, opened a new log
I20260926 06:57:01.240898 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 22746
I20260926 06:57:01.240998 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:57:01.241199 767 external_mini_cluster.cc:1428] Setting key 4cc641661c17dd6b8afc4908c4a41f2e
I20260926 06:57:01.241191 22814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303681c45a1245339842dbabb21ab04e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 46023 } }
I20260926 06:57:01.241422 22814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:57:01.241477 22814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 303681c45a1245339842dbabb21ab04e, State: Initialized, Role: FOLLOWER
I20260926 06:57:01.241647 22814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303681c45a1245339842dbabb21ab04e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 46023 } }
I20260926 06:57:01.241742 22814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:57:01.241804 22814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:57:01.241860 22814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:57:01.242981 22814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303681c45a1245339842dbabb21ab04e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 46023 } }
I20260926 06:57:01.243090 22814 leader_election.cc:302] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 303681c45a1245339842dbabb21ab04e; no voters:
I20260926 06:57:01.243233 22814 leader_election.cc:288] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:57:01.243283 22819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:57:01.243474 22814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:57:01.243471 22819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [term 1 LEADER]: Becoming Leader. State: Replica: 303681c45a1245339842dbabb21ab04e, State: Running, Role: LEADER
I20260926 06:57:01.243700 22819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303681c45a1245339842dbabb21ab04e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 46023 } }
I20260926 06:57:01.244086 22819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 303681c45a1245339842dbabb21ab04e. Latest consensus state: current_term: 1 leader_uuid: "303681c45a1245339842dbabb21ab04e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303681c45a1245339842dbabb21ab04e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 46023 } } }
I20260926 06:57:01.244107 22820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "303681c45a1245339842dbabb21ab04e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303681c45a1245339842dbabb21ab04e" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 46023 } } }
I20260926 06:57:01.244400 22819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [sys.catalog]: This master's current role is: LEADER
I20260926 06:57:01.244406 22820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e [sys.catalog]: This master's current role is: LEADER
W20260926 06:57:01.245585 22834 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:57:01.245651 22834 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:57:01.245733 22829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:57:01.245903 22829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:57:01.247293 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:01.241745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42805 (local address 127.0.191.254:46023)
0926 06:57:01.241964 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:01.241972 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:01.242020 (+ 48us) server_negotiation.cc:407] Connection header received
0926 06:57:01.242138 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:01.242144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:01.242214 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:01.242374 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:01.242534 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:01.243940 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:01.245214 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:01.245828 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:01.245939 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:01.246623 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:01.246626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:01.246626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:57:01.246824 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:01.246861 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:57:01.246876 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:01.246943 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:01.247026 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 06:57:01.247088 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":93,"threads_started":1}
I20260926 06:57:01.247619 22829 catalog_manager.cc:1347] Generated new cluster ID: 94936793bcf34301a8d2616a9197dbc1
I20260926 06:57:01.247669 22829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:57:01.248301 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:57:01.253109 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:01.249797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34709 (local address 127.0.191.254:46023)
0926 06:57:01.249828 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:01.249830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:01.249840 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:57:01.249903 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:01.249904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:01.249915 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:01.249966 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:01.250085 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:01.250787 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:01.251783 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:01.252130 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:01.252209 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:01.252861 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:01.252863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:01.252863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:57:01.252869 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:01.252886 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:57:01.252890 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:01.252950 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:01.253013 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 06:57:01.253036 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 06:57:01.258322 22829 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:57:01.258841 22829 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:57:01.265437 22829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 303681c45a1245339842dbabb21ab04e: Generated new TSK 0
I20260926 06:57:01.265607 22829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:57:01.268818 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 22746
2026-09-26T06:57:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:57:01 dist-test-slave-ws74 krb5kdc[22845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:57:01 dist-test-slave-ws74 krb5kdc[22845](info): set up 2 sockets
Sep 26 06:57:01 dist-test-slave-ws74 krb5kdc[22845](info): commencing operation
krb5kdc: starting...
W20260926 06:57:02.293607 767 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:57:02 dist-test-slave-ws74 krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:57:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:57:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:57:02.410393 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:39297
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:43695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:39297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:57:02.493191 22858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:57:02.493364 22858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:57:02.493409 22858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:57:02.494526 22858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:57:02.494566 22858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:57:02.494580 22858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:57:02.494591 22858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:57:02.494638 22858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:57:02.496027 22858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:43695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:39297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:39297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:57:02.496258 22858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:57:02.496474 22858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:57:02.501124 22866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:02.501133 22864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:57:02.501243 22858 server_base.cc:1034] running on GCE node
W20260926 06:57:02.501133 22863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:57:02.501610 22858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:57:02.501869 22858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:57:02.503044 22858 hybrid_clock.cc:648] HybridClock initialized: now 1790405822503028 us; error 28 us; skew 500 ppm
Sep 26 06:57:02 dist-test-slave-ws74 krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405822, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:57:02.505414 22858 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:57:02.505784 22858 webserver.cc:460] Webserver started at http://127.0.191.254:37607/ using document root <none> and password file <none>
I20260926 06:57:02.505967 22858 fs_manager.cc:359] Metadata directory not provided
I20260926 06:57:02.506006 22858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:57:02.506100 22858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:57:02.507052 22858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "fab7b1b19f85456f92de2a979f676ccf"
format_stamp: "Formatted at 2026-09-26 06:57:02 on dist-test-slave-ws74"
server_key: "b18b246c444c61b4cc2459838b4f374f"
server_key_iv: "94f78da14c45818c09609dcf450f6363"
server_key_version: "encryptionkey@0"
I20260926 06:57:02.507331 22858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "fab7b1b19f85456f92de2a979f676ccf"
format_stamp: "Formatted at 2026-09-26 06:57:02 on dist-test-slave-ws74"
server_key: "b18b246c444c61b4cc2459838b4f374f"
server_key_iv: "94f78da14c45818c09609dcf450f6363"
server_key_version: "encryptionkey@0"
I20260926 06:57:02.508507 22858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:57:02.509155 22873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:57:02.509303 22858 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:57:02.509434 22858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "fab7b1b19f85456f92de2a979f676ccf"
format_stamp: "Formatted at 2026-09-26 06:57:02 on dist-test-slave-ws74"
server_key: "b18b246c444c61b4cc2459838b4f374f"
server_key_iv: "94f78da14c45818c09609dcf450f6363"
server_key_version: "encryptionkey@0"
I20260926 06:57:02.509506 22858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:57:02.512624 22858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:57:02.514489 22858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:57:02.514591 22858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:57:02.518185 22858 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:39297
I20260926 06:57:02.518239 22925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:39297 every 8 connection(s)
I20260926 06:57:02.518541 22858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:57:02.519505 22926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:57:02.521657 22926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf: Bootstrap starting.
I20260926 06:57:02.522220 22926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf: Neither blocks nor log segments found. Creating new log.
I20260926 06:57:02.522500 22926 log.cc:824] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf: Log is configured to *not* fsync() on all Append() calls
I20260926 06:57:02.523203 22926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf: No bootstrap required, opened a new log
I20260926 06:57:02.524734 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 22858
I20260926 06:57:02.524825 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:57:02.524765 22926 raft_consensus.cc:348] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab7b1b19f85456f92de2a979f676ccf" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39297 } }
I20260926 06:57:02.524997 22926 raft_consensus.cc:374] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:57:02.525055 22926 raft_consensus.cc:729] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab7b1b19f85456f92de2a979f676ccf, State: Initialized, Role: FOLLOWER
I20260926 06:57:02.525107 767 external_mini_cluster.cc:1428] Setting key 9ba10e466e664b9ee60e73a9a1651d65
I20260926 06:57:02.525209 22926 consensus_queue.cc:260] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab7b1b19f85456f92de2a979f676ccf" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39297 } }
I20260926 06:57:02.525315 22926 raft_consensus.cc:388] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:57:02.525359 22926 raft_consensus.cc:482] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:57:02.525460 22926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:57:02.526381 22926 raft_consensus.cc:504] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab7b1b19f85456f92de2a979f676ccf" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39297 } }
I20260926 06:57:02.526491 22926 leader_election.cc:302] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fab7b1b19f85456f92de2a979f676ccf; no voters:
I20260926 06:57:02.526654 22926 leader_election.cc:288] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:57:02.526705 22931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:57:02.526835 22931 raft_consensus.cc:686] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [term 1 LEADER]: Becoming Leader. State: Replica: fab7b1b19f85456f92de2a979f676ccf, State: Running, Role: LEADER
I20260926 06:57:02.526898 22926 sys_catalog.cc:564] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:57:02.527004 22931 consensus_queue.cc:237] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab7b1b19f85456f92de2a979f676ccf" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39297 } }
I20260926 06:57:02.527376 22930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:02.525626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57507 (local address 127.0.191.254:39297)
0926 06:57:02.525818 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:02.525828 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:02.525859 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:57:02.525981 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:02.525986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:02.526008 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:02.526139 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:02.526892 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:02.526895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:02.526895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:57:02.526900 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:02.526945 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:57:02.526960 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:02.527000 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:02.527137 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 06:57:02.527179 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260926 06:57:02.527433 22932 sys_catalog.cc:455] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fab7b1b19f85456f92de2a979f676ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab7b1b19f85456f92de2a979f676ccf" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39297 } } }
I20260926 06:57:02.527499 22932 sys_catalog.cc:458] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [sys.catalog]: This master's current role is: LEADER
I20260926 06:57:02.528138 22933 sys_catalog.cc:455] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [sys.catalog]: SysCatalogTable state changed. Reason: New leader fab7b1b19f85456f92de2a979f676ccf. Latest consensus state: current_term: 1 leader_uuid: "fab7b1b19f85456f92de2a979f676ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab7b1b19f85456f92de2a979f676ccf" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 39297 } } }
I20260926 06:57:02.528218 22933 sys_catalog.cc:458] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf [sys.catalog]: This master's current role is: LEADER
I20260926 06:57:02.528714 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260926 06:57:02.528723 22947 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:57:02.528786 22947 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:57:02.528831 22939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:57:02.528980 22939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:57:02.530524 22939 catalog_manager.cc:1347] Generated new cluster ID: 04a0ed208e4b48d1ace129425fe12280
I20260926 06:57:02.530586 22939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:57:02.531342 22930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:02.530257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33353 (local address 127.0.191.254:39297)
0926 06:57:02.530289 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:02.530291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:02.530433 (+ 142us) server_negotiation.cc:407] Connection header received
0926 06:57:02.530480 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:02.530481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:02.530486 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:02.530533 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:02.531141 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:02.531143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:02.531143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:57:02.531146 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:02.531160 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:57:02.531164 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:02.531218 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:02.531276 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 06:57:02.531287 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:57:02.537352 22939 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:57:02.537519 22939 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:57:02.547083 22939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fab7b1b19f85456f92de2a979f676ccf: Generated new TSK 0
I20260926 06:57:02.547261 22939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:57:02.553761 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 22858
2026-09-26T06:57:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1285 ms)
[----------] 18 tests from AuthTokenIssuingTest (23682 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:57:02 dist-test-slave-ws74 krb5kdc[22957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:57:02 dist-test-slave-ws74 krb5kdc[22957](info): set up 2 sockets
Sep 26 06:57:02 dist-test-slave-ws74 krb5kdc[22957](info): commencing operation
krb5kdc: starting...
W20260926 06:57:03.577634 767 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:57:03 dist-test-slave-ws74 krb5kdc[22957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:57:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:57:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:57:03.696568 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:40631
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:36375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:40631,127.0.191.253:41035,127.0.191.252:42951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:57:03.777426 22970 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 06:57:03.777630 22970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:57:03.777647 22970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:57:03.777667 22970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:57:03.778810 22970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:57:03.778856 22970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:57:03.778872 22970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:57:03.778908 22970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:57:03.778940 22970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:57:03.780380 22970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:36375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:40631,127.0.191.253:41035,127.0.191.252:42951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:40631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:57:03.780625 22970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:57:03.780853 22970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:57:03.785706 22976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:03.785709 22978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:03.785728 22975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:57:03.785924 22970 server_base.cc:1034] running on GCE node
I20260926 06:57:03.786190 22970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:57:03.786535 22970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:57:03.787705 22970 hybrid_clock.cc:648] HybridClock initialized: now 1790405823787686 us; error 37 us; skew 500 ppm
Sep 26 06:57:03 dist-test-slave-ws74 krb5kdc[22957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:57:03.790325 22970 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:57:03.790825 22970 webserver.cc:460] Webserver started at http://127.0.191.254:33121/ using document root <none> and password file <none>
I20260926 06:57:03.791072 22970 fs_manager.cc:359] Metadata directory not provided
I20260926 06:57:03.791143 22970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:57:03.791285 22970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:57:03.792354 22970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "25e9283e628a40b2bfcea84f045f3983"
format_stamp: "Formatted at 2026-09-26 06:57:03 on dist-test-slave-ws74"
server_key: "92de4ab0742d5757d9caed5d1cac650d"
server_key_iv: "df71c24ecf3512eb0d33bedd12f0a4b1"
server_key_version: "encryptionkey@0"
I20260926 06:57:03.792723 22970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "25e9283e628a40b2bfcea84f045f3983"
format_stamp: "Formatted at 2026-09-26 06:57:03 on dist-test-slave-ws74"
server_key: "92de4ab0742d5757d9caed5d1cac650d"
server_key_iv: "df71c24ecf3512eb0d33bedd12f0a4b1"
server_key_version: "encryptionkey@0"
I20260926 06:57:03.794108 22970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:57:03.794806 22985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:57:03.794975 22970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:57:03.795066 22970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "25e9283e628a40b2bfcea84f045f3983"
format_stamp: "Formatted at 2026-09-26 06:57:03 on dist-test-slave-ws74"
server_key: "92de4ab0742d5757d9caed5d1cac650d"
server_key_iv: "df71c24ecf3512eb0d33bedd12f0a4b1"
server_key_version: "encryptionkey@0"
I20260926 06:57:03.795147 22970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:57:03.815251 22970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:57:03.817689 22970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:57:03.817855 22970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:57:03.821897 22970 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:40631
I20260926 06:57:03.821931 23037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:40631 every 8 connection(s)
I20260926 06:57:03.822283 22970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:57:03.823232 23038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:57:03.824754 23038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:57:03 dist-test-slave-ws74 krb5kdc[22957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:57:03.831997 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 22970
I20260926 06:57:03.832121 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:57:03.832312 767 external_mini_cluster.cc:1428] Setting key b8f4609a5e077d7df3e0c77736864f27
I20260926 06:57:03.834547 23041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.825536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45246 (local address 127.0.191.254:40631)
0926 06:57:03.825667 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:03.825676 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:03.825791 (+ 115us) server_negotiation.cc:407] Connection header received
0926 06:57:03.826344 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:03.826351 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:03.826368 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:03.826466 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:03.826973 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:03.827771 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:03.828628 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:03.828924 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:03.829051 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:03.831008 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:03.831015 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:03.831015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:03.831022 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:03.832738 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:03.833159 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:03.833162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:03.833162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:03.833268 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:03.833572 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:03.833573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:03.833573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:03.833875 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:03.833940 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:03.834293 (+ 353us) server_negotiation.cc:299] Negotiation successful
0926 06:57:03.834385 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260926 06:57:03.834666 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.825432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:40631 (local address 127.0.0.1:45246)
0926 06:57:03.825648 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:03.825693 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:03.826182 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:03.826472 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:03.826474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:03.826733 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:03.826874 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:03.826878 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:03.827874 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:03.827875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:03.828529 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:03.828530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:03.829043 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:03.829044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:03.829106 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:03.829621 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:03.829629 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:03.830920 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:03.832858 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:03.832864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:03.832866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:03.833071 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:03.833351 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:03.833351 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:03.833352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:03.833503 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:03.834054 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:03.834061 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:03.834143 (+ 82us) client_negotiation.cc:769] Sending connection context
0926 06:57:03.834239 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 06:57:03.834521 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":106,"thread_start_us":108,"threads_started":1}
I20260926 06:57:03.835373 23038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:57:03.835737 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.835612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:41035 (local address 127.0.0.1:54824)
0926 06:57:03.835660 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:03.835707 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:41035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 06:57:03.835896 22988 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.191.253:41035: connect: Connection refused (error 111)
I20260926 06:57:03.836339 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.836205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:41035 (local address 127.0.0.1:54836)
0926 06:57:03.836263 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:03.836295 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:41035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260926 06:57:03.836465 23038 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.253:41035: Network error: Client connection negotiation failed: client connection to 127.0.191.253:41035: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.0.191.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.253.keytab.
Entry for principal kudu/127.0.191.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.253.keytab.
I20260926 06:57:03.856345 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.253:41035
--webserver_interface=127.0.191.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.253.keytab
--principal=kudu/127.0.191.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:36375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:40631,127.0.191.253:41035,127.0.191.252:42951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:57:03.885703 23038 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } attempt: 1
I20260926 06:57:03.886236 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.886071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:41035 (local address 127.0.0.1:54846)
0926 06:57:03.886144 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:03.886195 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:41035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260926 06:57:03.886615 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.886519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:41035 (local address 127.0.0.1:54852)
0926 06:57:03.886565 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:03.886595 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:41035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 06:57:03.886775 23038 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.253:41035: Network error: Client connection negotiation failed: client connection to 127.0.191.253:41035: connect: Connection refused (error 111)
W20260926 06:57:03.935478 23045 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 06:57:03.935674 23045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:57:03.935691 23045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:57:03.935711 23045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:57:03.936844 23045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:57:03.936882 23045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:57:03.936896 23045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:57:03.936936 23045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:57:03.936966 23045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:57:03.938447 23045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:36375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:40631,127.0.191.253:41035,127.0.191.252:42951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.253.keytab
--principal=kudu/127.0.191.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.253:41035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.191.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:57:03.938688 23045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:57:03.938915 23045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:57:03.943555 23050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:03.943588 23053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:03.943552 23051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:57:03.943881 23045 server_base.cc:1034] running on GCE node
I20260926 06:57:03.944088 23045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:57:03.944305 23045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:57:03.945456 23045 hybrid_clock.cc:648] HybridClock initialized: now 1790405823945434 us; error 30 us; skew 500 ppm
Sep 26 06:57:03 dist-test-slave-ws74 krb5kdc[22957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:57:03.947857 23045 init.cc:377] Logged in from keytab as kudu/127.0.191.253@KRBTEST.COM (short username kudu)
I20260926 06:57:03.948279 23045 webserver.cc:460] Webserver started at http://127.0.191.253:33445/ using document root <none> and password file <none>
I20260926 06:57:03.948544 23045 fs_manager.cc:359] Metadata directory not provided
I20260926 06:57:03.948719 23045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:57:03.948899 23045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:57:03.950032 23045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/data/instance:
uuid: "382c230fa3d3422681abc8a0a48131f2"
format_stamp: "Formatted at 2026-09-26 06:57:03 on dist-test-slave-ws74"
server_key: "2dd430884d9938acd04dd7ab79d4d0c2"
server_key_iv: "2c690f1c36ded213c03271ce943a9fe7"
server_key_version: "encryptionkey@0"
I20260926 06:57:03.950378 23045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/wal/instance:
uuid: "382c230fa3d3422681abc8a0a48131f2"
format_stamp: "Formatted at 2026-09-26 06:57:03 on dist-test-slave-ws74"
server_key: "2dd430884d9938acd04dd7ab79d4d0c2"
server_key_iv: "2c690f1c36ded213c03271ce943a9fe7"
server_key_version: "encryptionkey@0"
I20260926 06:57:03.951678 23045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:57:03.952395 23060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:57:03.952600 23045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:57:03.952698 23045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/wal
uuid: "382c230fa3d3422681abc8a0a48131f2"
format_stamp: "Formatted at 2026-09-26 06:57:03 on dist-test-slave-ws74"
server_key: "2dd430884d9938acd04dd7ab79d4d0c2"
server_key_iv: "2c690f1c36ded213c03271ce943a9fe7"
server_key_version: "encryptionkey@0"
I20260926 06:57:03.952777 23045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:57:03.954945 23038 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } attempt: 2
I20260926 06:57:03.955320 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.955201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:41035 (local address 127.0.0.1:54866)
0926 06:57:03.955256 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:03.955294 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:41035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 06:57:03.955662 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.955567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:41035 (local address 127.0.0.1:54882)
0926 06:57:03.955612 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:03.955636 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:41035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 06:57:03.955745 23038 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.253:41035: Network error: Client connection negotiation failed: client connection to 127.0.191.253:41035: connect: Connection refused (error 111)
I20260926 06:57:03.966262 23045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:57:03.968096 23045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:57:03.968253 23045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:57:03.972424 23045 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.253:41035
I20260926 06:57:03.972476 23112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.253:41035 every 8 connection(s)
I20260926 06:57:03.972819 23045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/data/info.pb
I20260926 06:57:03.973899 23113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:57:03.975306 23113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:57:03.980777 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 23045
I20260926 06:57:03.980855 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-1/wal/instance
Sep 26 06:57:03 dist-test-slave-ws74 krb5kdc[22957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.253@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:57:03.981073 767 external_mini_cluster.cc:1428] Setting key 07fe1aa267b31286fa67fd8153fefae8
I20260926 06:57:03.984449 23041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.976084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45252 (local address 127.0.191.254:40631)
0926 06:57:03.976161 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:03.976163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:03.976323 (+ 160us) server_negotiation.cc:407] Connection header received
0926 06:57:03.977075 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:03.977077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:03.977090 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:03.977164 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:03.977802 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:03.978273 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:03.979180 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:03.979359 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:03.979408 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:03.981476 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:03.981477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:03.981478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:03.981485 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:03.982914 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:03.983405 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:03.983407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:03.983407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:03.983461 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:03.983769 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:03.983770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:03.983771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:03.983901 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:03.983965 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:03.984308 (+ 343us) server_negotiation.cc:299] Negotiation successful
0926 06:57:03.984380 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 06:57:03.984704 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.975956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:40631 (local address 127.0.0.1:45252)
0926 06:57:03.976189 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:03.976230 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:03.976932 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:03.977221 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:03.977224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:03.977530 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:03.977719 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:03.977723 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:03.978350 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:03.978350 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:03.979112 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:03.979113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:03.979448 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:03.979449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:03.979554 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:03.979965 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:03.979973 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:03.981357 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:03.983017 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:03.983024 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:03.983026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:03.983334 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:03.983530 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:03.983530 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:03.983531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:03.983680 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:03.983978 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:03.983984 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:03.984143 (+ 159us) client_negotiation.cc:769] Sending connection context
0926 06:57:03.984257 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 06:57:03.984434 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":72,"threads_started":1}
I20260926 06:57:03.985258 23113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.191.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.252@KRBTEST.COM" created.
Sep 26 06:57:03 dist-test-slave-ws74 krb5kdc[22957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.253@KRBTEST.COM for kudu/127.0.191.253@KRBTEST.COM
I20260926 06:57:03.995669 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.985572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:41035 (local address 127.0.0.1:54886)
0926 06:57:03.985630 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:03.985643 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:03.985691 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:03.985902 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:03.985903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:03.986064 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:03.986125 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:03.986126 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:03.986959 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:03.986960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:03.987811 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:03.987813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:03.988252 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:03.988254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:03.988336 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:03.988678 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:03.988683 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:03.992394 (+ 3711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:03.994481 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:03.994482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:03.994482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:03.994702 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:03.994904 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:03.994904 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:03.994905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:03.994973 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:03.995227 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:03.995228 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:03.995275 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 06:57:03.995356 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 06:57:03.995487 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 06:57:03.995673 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.985608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54886 (local address 127.0.191.253:41035)
0926 06:57:03.985755 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:03.985759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:03.985776 (+ 17us) server_negotiation.cc:407] Connection header received
0926 06:57:03.985803 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:03.985805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:03.985815 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:03.985878 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:03.986202 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:03.986865 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:03.987900 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:03.988180 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:03.988227 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:03.992475 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:03.992482 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:03.992482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:03.992488 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:03.994403 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:03.994786 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:03.994789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:03.994789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:03.994838 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:03.995031 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:03.995039 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:03.995040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:03.995160 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:03.995219 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:03.995395 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 06:57:03.995569 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":120,"thread_start_us":95,"threads_started":1}
I20260926 06:57:03.996353 23113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:57:03.996760 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.996598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:42951 (local address 127.0.0.1:37208)
0926 06:57:03.996660 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:03.996710 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 06:57:03.996920 23061 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
I20260926 06:57:03.997336 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:03.997237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:42951 (local address 127.0.0.1:37212)
0926 06:57:03.997275 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:03.997306 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260926 06:57:03.997457 23113 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:42951: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.252.keytab.
Entry for principal kudu/127.0.191.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.252.keytab.
I20260926 06:57:04.007761 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.252:42951
--webserver_interface=127.0.191.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.252.keytab
--principal=kudu/127.0.191.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:36375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:40631,127.0.191.253:41035,127.0.191.252:42951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:57:04.046722 23113 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } attempt: 1
I20260926 06:57:04.046926 23038 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } attempt: 3
I20260926 06:57:04.047257 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.047096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:42951 (local address 127.0.0.1:37216)
0926 06:57:04.047167 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.047219 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 06:57:04.047647 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.047512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:42951 (local address 127.0.0.1:37224)
0926 06:57:04.047579 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.047612 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 06:57:04.047758 23113 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:42951: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
Sep 26 06:57:04 dist-test-slave-ws74 krb5kdc[22957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for kudu/127.0.191.253@KRBTEST.COM
I20260926 06:57:04.054056 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.047189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54902 (local address 127.0.191.253:41035)
0926 06:57:04.047226 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.047228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.047266 (+ 38us) server_negotiation.cc:407] Connection header received
0926 06:57:04.047363 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.047364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.047378 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.047450 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.047873 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.048358 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.049081 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.049307 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.049407 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.051552 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:04.051553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:04.051553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:04.051559 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:04.052968 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.053339 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.053340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.053340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.053435 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.053624 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.053625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.053625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.053727 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:04.053779 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:04.053937 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 06:57:04.053996 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:57:04.054107 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.047173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:41035 (local address 127.0.0.1:54902)
0926 06:57:04.047222 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.047233 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:04.047282 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:04.047462 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:04.047463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:04.047698 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:04.047780 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:04.047783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.048458 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.048460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:04.048990 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:04.048992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.049413 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.049415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:04.049493 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.049988 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:04.049992 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:04.051458 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:04.053050 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:04.053051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:04.053052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:04.053267 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:04.053496 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:04.053497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:04.053498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:04.053563 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:04.053789 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:04.053790 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:04.053841 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 06:57:04.053919 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 06:57:04.054015 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 06:57:04.054802 23038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:57:04.055169 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.055062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:42951 (local address 127.0.0.1:37226)
0926 06:57:04.055111 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.055143 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 06:57:04.055505 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.055433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:42951 (local address 127.0.0.1:37228)
0926 06:57:04.055462 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.055488 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260926 06:57:04.055601 23038 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:42951: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
I20260926 06:57:04.086755 23038 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } attempt: 1
I20260926 06:57:04.087406 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.087271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:42951 (local address 127.0.0.1:37236)
0926 06:57:04.087332 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.087365 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 06:57:04.087846 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.087650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:42951 (local address 127.0.0.1:37246)
0926 06:57:04.087790 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.087816 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
W20260926 06:57:04.088021 23038 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:42951: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
W20260926 06:57:04.091559 23120 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 06:57:04.091741 23120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:57:04.091784 23120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:57:04.091822 23120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:57:04.092929 23120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:57:04.092968 23120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:57:04.092983 23120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:57:04.092995 23120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:57:04.093007 23120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:57:04.094497 23120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:36375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:40631,127.0.191.253:41035,127.0.191.252:42951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/krb5kdc/kudu_127.0.191.252.keytab
--principal=kudu/127.0.191.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.252:42951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.191.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:57:04.094729 23120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:57:04.094964 23120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:57:04.099642 23129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:04.099651 23127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:57:04.099767 23120 server_base.cc:1034] running on GCE node
W20260926 06:57:04.099660 23126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:57:04.100090 23120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:57:04.100324 23120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:57:04.101435 23120 hybrid_clock.cc:648] HybridClock initialized: now 1790405824101416 us; error 46 us; skew 500 ppm
Sep 26 06:57:04 dist-test-slave-ws74 krb5kdc[22957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405824, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:57:04.103768 23120 init.cc:377] Logged in from keytab as kudu/127.0.191.252@KRBTEST.COM (short username kudu)
I20260926 06:57:04.104171 23120 webserver.cc:460] Webserver started at http://127.0.191.252:37071/ using document root <none> and password file <none>
I20260926 06:57:04.104400 23120 fs_manager.cc:359] Metadata directory not provided
I20260926 06:57:04.104446 23120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:57:04.104569 23120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:57:04.105633 23120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/data/instance:
uuid: "f22de34b18eb453fbd90e2d4637e03a8"
format_stamp: "Formatted at 2026-09-26 06:57:04 on dist-test-slave-ws74"
server_key: "a9a55412d8957950c21c8466b325ab06"
server_key_iv: "ad9c98d3ee73fae9fa3a9b8dd090fb32"
server_key_version: "encryptionkey@0"
I20260926 06:57:04.105983 23120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/wal/instance:
uuid: "f22de34b18eb453fbd90e2d4637e03a8"
format_stamp: "Formatted at 2026-09-26 06:57:04 on dist-test-slave-ws74"
server_key: "a9a55412d8957950c21c8466b325ab06"
server_key_iv: "ad9c98d3ee73fae9fa3a9b8dd090fb32"
server_key_version: "encryptionkey@0"
I20260926 06:57:04.107254 23120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:57:04.107995 23136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:57:04.108182 23120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:57:04.108285 23120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/wal
uuid: "f22de34b18eb453fbd90e2d4637e03a8"
format_stamp: "Formatted at 2026-09-26 06:57:04 on dist-test-slave-ws74"
server_key: "a9a55412d8957950c21c8466b325ab06"
server_key_iv: "ad9c98d3ee73fae9fa3a9b8dd090fb32"
server_key_version: "encryptionkey@0"
I20260926 06:57:04.108358 23120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:57:04.115903 23113 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } attempt: 2
I20260926 06:57:04.116303 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.116174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:42951 (local address 127.0.0.1:37250)
0926 06:57:04.116235 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.116273 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 06:57:04.116331 23120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:57:04.116712 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.116608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:42951 (local address 127.0.0.1:37252)
0926 06:57:04.116654 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.116680 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 06:57:04.116799 23113 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:42951: Network error: Client connection negotiation failed: client connection to 127.0.191.252:42951: connect: Connection refused (error 111)
I20260926 06:57:04.118605 23120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:57:04.118710 23120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:57:04.122612 23120 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.252:42951
I20260926 06:57:04.122659 23188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.252:42951 every 8 connection(s)
I20260926 06:57:04.122999 23120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/data/info.pb
I20260926 06:57:04.123939 23189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:57:04.125458 23189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:57:04 dist-test-slave-ws74 krb5kdc[22957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405824, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.252@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:57:04.132592 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 23120
I20260926 06:57:04.132701 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790405559478510-767-0/minicluster-data/master-2/wal/instance
I20260926 06:57:04.132900 767 external_mini_cluster.cc:1428] Setting key 838f7e38f2bf537ae836ae4c990f812c
I20260926 06:57:04.134508 23041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.126086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45258 (local address 127.0.191.254:40631)
0926 06:57:04.126131 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.126133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.126309 (+ 176us) server_negotiation.cc:407] Connection header received
0926 06:57:04.126731 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.126732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.126740 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.126794 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.127319 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.127791 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.128710 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.128884 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.128933 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.130909 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:04.130910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:04.130910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:04.130916 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:04.132890 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.133481 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.133483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.133484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.133549 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.133847 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.133848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.133849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.133994 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:04.134092 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:04.134315 (+ 223us) server_negotiation.cc:299] Negotiation successful
0926 06:57:04.134392 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:57:04.134622 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.126043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:40631 (local address 127.0.0.1:45258)
0926 06:57:04.126192 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.126229 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:04.126655 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:04.126851 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:04.126854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:04.127072 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:04.127253 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:04.127257 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.127874 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.127875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:04.128638 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:04.128640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.128966 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.128967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:04.129065 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.129540 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:04.129547 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:04.130828 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:04.132975 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:04.132981 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:04.132982 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:04.133327 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:04.133643 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:04.133644 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:04.133644 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:04.133778 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:04.134082 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:04.134086 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:04.134188 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 06:57:04.134266 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 06:57:04.134373 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260926 06:57:04.135172 23189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:57:04 dist-test-slave-ws74 krb5kdc[22957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.252@KRBTEST.COM
Sep 26 06:57:04 dist-test-slave-ws74 krb5kdc[22957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405824, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.252@KRBTEST.COM for kudu/127.0.191.253@KRBTEST.COM
I20260926 06:57:04.142149 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.135473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54910 (local address 127.0.191.253:41035)
0926 06:57:04.135511 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.135513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.135521 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:57:04.135545 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.135546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.135557 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.135615 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.135883 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.136443 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.137319 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.137634 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.137688 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.139571 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:04.139572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:04.139572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:04.139579 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:04.140996 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.141355 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.141355 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.141356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.141485 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.141647 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.141648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.141648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.141773 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:04.141836 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:04.141981 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 06:57:04.142054 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:57:04.142168 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.135400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:41035 (local address 127.0.0.1:54910)
0926 06:57:04.135445 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.135456 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:04.135491 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:04.135640 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:04.135640 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:04.135764 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:04.135803 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:04.135804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.136519 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.136519 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:04.137237 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:04.137240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.137710 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.137712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:04.137790 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.138114 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:04.138119 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:04.139492 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:04.141072 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:04.141073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:04.141073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:04.141290 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:04.141529 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:04.141529 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:04.141529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:04.141578 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:04.141841 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:04.141842 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:04.141880 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 06:57:04.141951 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 06:57:04.142046 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 06:57:04.142272 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.133304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37254 (local address 127.0.191.252:42951)
0926 06:57:04.133516 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.133520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.133648 (+ 128us) server_negotiation.cc:407] Connection header received
0926 06:57:04.133683 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.133686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.133699 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.133768 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.134496 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.135534 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.136243 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.136671 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.136721 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.139160 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:04.139168 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:04.139168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:04.139175 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:04.140749 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.141112 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.141114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.141114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.141164 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.141549 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.141550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.141550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.141725 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:04.141877 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:04.141973 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 06:57:04.142113 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":187,"thread_start_us":121,"threads_started":1}
I20260926 06:57:04.142518 23189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:57:04 dist-test-slave-ws74 krb5kdc[22957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405824, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.252@KRBTEST.COM for kudu/127.0.191.252@KRBTEST.COM
Sep 26 06:57:04 dist-test-slave-ws74 krb5kdc[22957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:57:04.148901 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.142735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:42951 (local address 127.0.0.1:37270)
0926 06:57:04.142785 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.142795 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:04.142834 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:04.142962 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:04.142963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:04.143118 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:04.143153 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:04.143155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.143857 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.143858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:04.144430 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:04.144431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.144750 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.144750 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:04.144801 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.145168 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:04.145191 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:04.146370 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:04.147909 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:04.147910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:04.147910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:04.148139 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:04.148341 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:04.148342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:04.148342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:04.148393 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:04.148676 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:04.148678 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:04.148724 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 06:57:04.148777 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 06:57:04.148842 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 06:57:04.149085 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.142784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37270 (local address 127.0.191.252:42951)
0926 06:57:04.142818 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.142820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.142828 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:57:04.142895 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.142895 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.142903 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.142955 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.143227 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.143783 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.144505 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.144683 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.144732 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.146468 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:04.146469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:04.146470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:04.146476 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:04.147815 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.148213 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.148214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.148215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.148273 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.148456 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.148457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.148457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.148583 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:04.148667 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:04.148825 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 06:57:04.148906 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:57:04.150511 23041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.142938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45266 (local address 127.0.191.254:40631)
0926 06:57:04.142978 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.142979 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.142987 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:57:04.143030 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.143031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.143045 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.143084 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.143558 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.144111 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.144941 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.145235 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.145284 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.147661 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:04.147663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:04.147663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:04.147670 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:04.149482 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.149790 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.149791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.149791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.149845 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.150011 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.150012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.150012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.150129 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:04.150293 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:04.150331 (+ 38us) server_negotiation.cc:299] Negotiation successful
0926 06:57:04.150422 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:57:04.150699 23189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8: Bootstrap starting.
I20260926 06:57:04.151573 23189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8: Neither blocks nor log segments found. Creating new log.
I20260926 06:57:04.151948 23189 log.cc:824] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8: Log is configured to *not* fsync() on all Append() calls
I20260926 06:57:04.152709 23189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8: No bootstrap required, opened a new log
I20260926 06:57:04.154326 23189 raft_consensus.cc:348] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } }
I20260926 06:57:04.154470 23189 raft_consensus.cc:729] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22de34b18eb453fbd90e2d4637e03a8, State: Initialized, Role: FOLLOWER
I20260926 06:57:04.154589 23189 consensus_queue.cc:260] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } }
I20260926 06:57:04.154919 23189 sys_catalog.cc:564] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:57:04.154870 23195 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } } }
I20260926 06:57:04.154981 23195 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8 [sys.catalog]: This master's current role is: FOLLOWER
Sep 26 06:57:04 dist-test-slave-ws74 krb5kdc[22957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.253@KRBTEST.COM
W20260926 06:57:04.157459 23206 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:57:04.157568 23206 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:57:04.159042 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.151119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54918 (local address 127.0.191.253:41035)
0926 06:57:04.151182 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.151184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.151192 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:57:04.151212 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.151213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.151221 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.151277 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.151760 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.152422 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.153227 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.153548 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.153623 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.155914 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:04.155915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:04.155915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:04.155921 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:04.157578 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.157954 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.157955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.157955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.158009 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.158192 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.158192 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.158193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.158306 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:04.158452 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:04.158859 (+ 407us) server_negotiation.cc:299] Negotiation successful
0926 06:57:04.158953 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 06:57:04.160408 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:57:04.157466 23120 master_runner.cc:186] Error getting master registration for 127.0.191.254:40631: OK, instance_id { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" instance_seqno: 1790405823820729 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 06:57:04.160583 23120 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 06:57:04.161322 23012 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "25e9283e628a40b2bfcea84f045f3983"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45266
I20260926 06:57:04.163180 23038 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } attempt: 2
Sep 26 06:57:04 dist-test-slave-ws74 krb5kdc[22957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for kudu/127.0.191.252@KRBTEST.COM
I20260926 06:57:04.169936 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.163502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37282 (local address 127.0.191.252:42951)
0926 06:57:04.163553 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.163555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.163562 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:57:04.163598 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.163598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.163606 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.163666 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.163926 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.164365 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.165012 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.165213 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.165293 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.167183 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:04.167185 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:04.167185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:04.167192 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:04.168753 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.169150 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.169151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.169151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.169204 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.169425 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.169426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.169427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.169569 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:04.169632 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:04.169773 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 06:57:04.169848 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:57:04.169940 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.163421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:42951 (local address 127.0.0.1:37282)
0926 06:57:04.163463 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.163474 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:04.163521 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:04.163685 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:04.163686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:04.163821 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:04.163853 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:04.163854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.164450 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.164451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:04.164932 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:04.164934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.165291 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.165292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:04.165347 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.165808 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:04.165814 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:04.167086 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:04.168850 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:04.168851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:04.168851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:04.169074 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:04.169273 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:04.169274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:04.169274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:04.169338 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:04.169638 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:04.169639 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:04.169684 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 06:57:04.169756 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:57:04.169844 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260926 06:57:04.171590 23038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983: Bootstrap starting.
I20260926 06:57:04.172261 23038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983: Neither blocks nor log segments found. Creating new log.
I20260926 06:57:04.172554 23038 log.cc:824] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983: Log is configured to *not* fsync() on all Append() calls
I20260926 06:57:04.173357 23038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983: No bootstrap required, opened a new log
I20260926 06:57:04.174685 23038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } }
I20260926 06:57:04.174808 23038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e9283e628a40b2bfcea84f045f3983, State: Initialized, Role: FOLLOWER
I20260926 06:57:04.174899 23038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } }
I20260926 06:57:04.175203 23038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:57:04.175248 23012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 06:57:04.175171 23209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } } }
I20260926 06:57:04.175303 23012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:57:04.175426 23209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:57:04.176220 23012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } }
I20260926 06:57:04.176502 23012 leader_election.cc:288] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [CANDIDATE]: Term 1 election: Requested vote from peers 382c230fa3d3422681abc8a0a48131f2 (127.0.191.253:41035), f22de34b18eb453fbd90e2d4637e03a8 (127.0.191.252:42951)
I20260926 06:57:04.176752 23163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "25e9283e628a40b2bfcea84f045f3983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f22de34b18eb453fbd90e2d4637e03a8"
I20260926 06:57:04.176864 23163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:57:04.177757 23163 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e9283e628a40b2bfcea84f045f3983 in term 1.
I20260926 06:57:04.178372 22986 leader_election.cc:302] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e9283e628a40b2bfcea84f045f3983, f22de34b18eb453fbd90e2d4637e03a8; no voters:
I20260926 06:57:04.178486 23209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:57:04.178632 23209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [term 1 LEADER]: Becoming Leader. State: Replica: 25e9283e628a40b2bfcea84f045f3983, State: Running, Role: LEADER
I20260926 06:57:04.178716 23209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } }
I20260926 06:57:04.179106 23209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25e9283e628a40b2bfcea84f045f3983. Latest consensus state: current_term: 1 leader_uuid: "25e9283e628a40b2bfcea84f045f3983" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } } }
I20260926 06:57:04.179165 23209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [sys.catalog]: This master's current role is: LEADER
W20260926 06:57:04.180348 23225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:57:04.180405 23225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:57:04.179033 22970 master_runner.cc:186] Error getting master registration for 127.0.191.253:41035: OK, instance_id { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" instance_seqno: 1790405823971182 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 06:57:04.181020 22970 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 06:57:04.187848 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.177629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37286 (local address 127.0.191.252:42951)
0926 06:57:04.177677 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.177679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.177866 (+ 187us) server_negotiation.cc:407] Connection header received
0926 06:57:04.178020 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.178020 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.178045 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.178076 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.179354 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.180087 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.181681 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.181965 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.182003 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.184451 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:04.184465 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:04.184465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:04.184473 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:04.186315 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.186983 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.186984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.186985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.187042 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.187248 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.187249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.187250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.187375 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:04.187459 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:04.187664 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 06:57:04.187738 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 06:57:04.188102 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.178087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54924 (local address 127.0.191.253:41035)
0926 06:57:04.178125 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.178127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.178833 (+ 706us) server_negotiation.cc:407] Connection header received
0926 06:57:04.179041 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.179042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.179052 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.179097 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.180529 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.181254 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.182362 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.182657 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.182705 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.184281 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:04.184283 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:04.184283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:04.184290 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:04.186391 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.187058 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.187059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.187060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.187124 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.187409 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.187410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.187411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.187546 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:04.187626 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:04.187928 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 06:57:04.188004 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:57:04.190460 23230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:57:04.190667 23230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:57:04.193259 23041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.178398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.0.191.254:40631)
0926 06:57:04.178433 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.178435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.185505 (+ 7070us) server_negotiation.cc:407] Connection header received
0926 06:57:04.185605 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.185606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.185616 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.185646 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.186263 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.186962 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.187972 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.188291 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.188340 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.190361 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:04.190362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:04.190363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:04.190370 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:04.192100 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.192466 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.192467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.192467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.192516 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.192709 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.192710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.192710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.192842 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:04.192923 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:04.193098 (+ 175us) server_negotiation.cc:299] Negotiation successful
0926 06:57:04.193167 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:57:04.198486 23163 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8 [term 1 FOLLOWER]: Refusing update from remote peer 25e9283e628a40b2bfcea84f045f3983: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:57:04.198855 23228 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:57:04.200522 23195 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25e9283e628a40b2bfcea84f045f3983. Latest consensus state: current_term: 1 leader_uuid: "25e9283e628a40b2bfcea84f045f3983" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } } }
I20260926 06:57:04.200609 23195 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:57:04.201583 23209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "25e9283e628a40b2bfcea84f045f3983" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } } }
I20260926 06:57:04.201663 23209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [sys.catalog]: This master's current role is: LEADER
I20260926 06:57:04.201725 23195 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "25e9283e628a40b2bfcea84f045f3983" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } } }
I20260926 06:57:04.201802 23195 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:57:04.201977 23233 mvcc.cc:205] Tried to move back new op lower bound from 7333502255914512384 to 7333502255836606464. Current Snapshot: MvccSnapshot[applied={T|T < 7333502255914512384}]
I20260926 06:57:04.202346 23230 catalog_manager.cc:1347] Generated new cluster ID: 2ffb1bd54f814974ad02811312248f7c
I20260926 06:57:04.202394 23230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:57:04.207928 23113 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } attempt: 3
Sep 26 06:57:04 dist-test-slave-ws74 krb5kdc[22957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.253@KRBTEST.COM for kudu/127.0.191.252@KRBTEST.COM
I20260926 06:57:04.216576 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.208138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:42951 (local address 127.0.0.1:37290)
0926 06:57:04.208218 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:04.208228 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:04.208260 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:04.208424 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:04.208425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:04.208574 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:04.208616 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:04.208617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.209541 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.209542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:04.210326 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:04.210327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.211686 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.211687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:04.211752 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.212256 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:04.212261 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:04.213682 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:04.215551 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:04.215552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:04.215553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:04.215745 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:04.215958 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:04.215959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:04.215959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:04.216020 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:04.216296 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:04.216297 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:04.216342 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 06:57:04.216404 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 06:57:04.216484 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 06:57:04.217046 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:04.208236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37290 (local address 127.0.191.252:42951)
0926 06:57:04.208279 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.208281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.208291 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:57:04.208313 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.208313 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.208323 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.208381 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.208715 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.209451 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.211259 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:04.211554 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:04.211619 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:04.213777 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:04.213778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:04.213779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:04.213785 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:04.215456 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.215823 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.215824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.215825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.215885 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:04.216083 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:04.216083 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:04.216084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:04.216201 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:04.216247 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:04.216890 (+ 643us) server_negotiation.cc:299] Negotiation successful
0926 06:57:04.216968 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 06:57:04.218482 23113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2: Bootstrap starting.
I20260926 06:57:04.219129 23113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2: Neither blocks nor log segments found. Creating new log.
I20260926 06:57:04.219414 23113 log.cc:824] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2: Log is configured to *not* fsync() on all Append() calls
I20260926 06:57:04.221022 23113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2: No bootstrap required, opened a new log
I20260926 06:57:04.222813 23113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } }
I20260926 06:57:04.222926 23113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 382c230fa3d3422681abc8a0a48131f2, State: Initialized, Role: FOLLOWER
I20260926 06:57:04.222995 23113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } }
I20260926 06:57:04.223281 23113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:57:04.224578 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "25e9283e628a40b2bfcea84f045f3983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "382c230fa3d3422681abc8a0a48131f2"
I20260926 06:57:04.225015 23086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:57:04.225951 23086 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2 [term 1 FOLLOWER]: Refusing update from remote peer 25e9283e628a40b2bfcea84f045f3983: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:57:04.226490 23209 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [LEADER]: Connected to new peer: Peer: permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:57:04.226761 23087 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e9283e628a40b2bfcea84f045f3983 in term 1.
I20260926 06:57:04.229090 23249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25e9283e628a40b2bfcea84f045f3983" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } } }
I20260926 06:57:04.229182 23249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:57:04.229228 23249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "25e9283e628a40b2bfcea84f045f3983" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } } }
I20260926 06:57:04.229264 23249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:57:04.229516 23230 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:57:04.229597 23252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25e9283e628a40b2bfcea84f045f3983. Latest consensus state: current_term: 1 leader_uuid: "25e9283e628a40b2bfcea84f045f3983" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } } }
I20260926 06:57:04.230219 23230 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:57:04.230252 23252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:57:04.235844 23228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "25e9283e628a40b2bfcea84f045f3983" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e9283e628a40b2bfcea84f045f3983" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 40631 } } peers { permanent_uuid: "382c230fa3d3422681abc8a0a48131f2" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 41035 } } peers { permanent_uuid: "f22de34b18eb453fbd90e2d4637e03a8" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 42951 } } }
I20260926 06:57:04.235936 23228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983 [sys.catalog]: This master's current role is: LEADER
I20260926 06:57:04.236265 23255 catalog_manager.cc:1259] Loaded cluster ID: 2ffb1bd54f814974ad02811312248f7c
I20260926 06:57:04.236308 23255 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2: loading cluster ID for follower catalog manager: success
I20260926 06:57:04.237347 23230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25e9283e628a40b2bfcea84f045f3983: Generated new TSK 0
I20260926 06:57:04.237584 23230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:57:04.237871 23255 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2: acquiring CA information for follower catalog manager: success
W20260926 06:57:04.237988 23255 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260926 06:57:04.259451 23041 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:04.258122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.0.191.254:40631)
0926 06:57:04.258179 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.258181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.258787 (+ 606us) server_negotiation.cc:407] Connection header received
0926 06:57:04.258898 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.258899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.258910 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.258941 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.259365 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45276: BlockingRecv error: recv got EOF from 127.0.0.1:45276 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260926 06:57:04.260169 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:04.258512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54926 (local address 127.0.191.253:41035)
0926 06:57:04.258549 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.258551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.259681 (+ 1130us) server_negotiation.cc:407] Connection header received
0926 06:57:04.259776 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.259777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.259786 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.259815 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.260111 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54926: BlockingRecv error: recv got EOF from 127.0.0.1:54926 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 06:57:04.261173 23193 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:04.260513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37294 (local address 127.0.191.252:42951)
0926 06:57:04.260549 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.260551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.260668 (+ 117us) server_negotiation.cc:407] Connection header received
0926 06:57:04.260755 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.260757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.260766 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.260795 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.261115 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37294: BlockingRecv error: recv got EOF from 127.0.0.1:37294 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 06:57:04.263370 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:04.262648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54940 (local address 127.0.191.253:41035)
0926 06:57:04.262683 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.262685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.262876 (+ 191us) server_negotiation.cc:407] Connection header received
0926 06:57:04.262977 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.262978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.262994 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.263022 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.263305 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54940: BlockingRecv error: recv got EOF from 127.0.0.1:54940 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 06:57:04.266194 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:04.265488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54942 (local address 127.0.191.253:41035)
0926 06:57:04.265519 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.265520 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.265745 (+ 225us) server_negotiation.cc:407] Connection header received
0926 06:57:04.265834 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.265835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.265844 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.265871 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.266154 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54942: BlockingRecv error: recv got EOF from 127.0.0.1:54942 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 06:57:04.269874 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:04.269110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54956 (local address 127.0.191.253:41035)
0926 06:57:04.269138 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.269140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.269340 (+ 200us) server_negotiation.cc:407] Connection header received
0926 06:57:04.269504 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.269504 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.269513 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.269540 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.269835 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54956: BlockingRecv error: recv got EOF from 127.0.0.1:54956 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260926 06:57:04.275542 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:04.274813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54970 (local address 127.0.191.253:41035)
0926 06:57:04.274844 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.274845 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.275068 (+ 223us) server_negotiation.cc:407] Connection header received
0926 06:57:04.275158 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.275159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.275168 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.275195 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.275502 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54970: BlockingRecv error: recv got EOF from 127.0.0.1:54970 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 06:57:04.285262 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:04.284521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54982 (local address 127.0.191.253:41035)
0926 06:57:04.284551 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.284553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.284781 (+ 228us) server_negotiation.cc:407] Connection header received
0926 06:57:04.284871 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.284872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.284881 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.284911 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.285207 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54982: BlockingRecv error: recv got EOF from 127.0.0.1:54982 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 06:57:04.303237 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:04.302416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54992 (local address 127.0.191.253:41035)
0926 06:57:04.302456 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.302458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.302707 (+ 249us) server_negotiation.cc:407] Connection header received
0926 06:57:04.302806 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.302807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.302818 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.302848 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.303185 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54992: BlockingRecv error: recv got EOF from 127.0.0.1:54992 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 06:57:04.337641 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:04.336615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54998 (local address 127.0.191.253:41035)
0926 06:57:04.336662 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.336664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.336959 (+ 295us) server_negotiation.cc:407] Connection header received
0926 06:57:04.337068 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.337069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.337082 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.337113 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.337579 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54998: BlockingRecv error: recv got EOF from 127.0.0.1:54998 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260926 06:57:04.403699 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:04.402981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55006 (local address 127.0.191.253:41035)
0926 06:57:04.403050 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.403052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.403171 (+ 119us) server_negotiation.cc:407] Connection header received
0926 06:57:04.403221 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.403222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.403232 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.403284 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.403636 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55006: BlockingRecv error: recv got EOF from 127.0.0.1:55006 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260926 06:57:04.533636 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:04.532918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55016 (local address 127.0.191.253:41035)
0926 06:57:04.533004 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.533006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.533052 (+ 46us) server_negotiation.cc:407] Connection header received
0926 06:57:04.533119 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.533120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.533131 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.533182 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.533569 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55016: BlockingRecv error: recv got EOF from 127.0.0.1:55016 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260926 06:57:04.791553 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:04.790878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55028 (local address 127.0.191.253:41035)
0926 06:57:04.790953 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:04.790955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:04.790966 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:57:04.791066 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:04.791067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:04.791086 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:04.791153 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:04.791487 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55028: BlockingRecv error: recv got EOF from 127.0.0.1:55028 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 06:57:05.158023 23206 catalog_manager.cc:1259] Loaded cluster ID: 2ffb1bd54f814974ad02811312248f7c
I20260926 06:57:05.158134 23206 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8: loading cluster ID for follower catalog manager: success
I20260926 06:57:05.158917 23206 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8: acquiring CA information for follower catalog manager: success
I20260926 06:57:05.159231 23206 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f22de34b18eb453fbd90e2d4637e03a8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:57:05.238669 23255 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 382c230fa3d3422681abc8a0a48131f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:57:05.307099 23321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:05.304683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55042 (local address 127.0.191.253:41035)
0926 06:57:05.304807 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:05.304809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:05.304820 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:57:05.304874 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:05.304875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:05.304934 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:05.305007 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:57:05.305156 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:05.305715 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:05.306436 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:05.306627 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:05.306679 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:05.306814 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:57:05.306928 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:57:05.306986 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:05.307017 (+ 31us) server_negotiation.cc:299] Negotiation successful
0926 06:57:05.307034 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260926 06:57:05.311058 23327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:05.308833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.0.191.252:42951)
0926 06:57:05.308935 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:05.308936 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:05.308956 (+ 20us) server_negotiation.cc:407] Connection header received
0926 06:57:05.308989 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:05.308990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:05.309021 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:05.309083 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:57:05.309237 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:05.309771 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:05.310479 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:05.310652 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:05.310698 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:05.310809 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:57:05.310887 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:57:05.310933 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:05.310979 (+ 46us) server_negotiation.cc:299] Negotiation successful
0926 06:57:05.310993 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":59,"threads_started":1}
I20260926 06:57:05.315546 23321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:05.312593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55044 (local address 127.0.191.253:41035)
0926 06:57:05.312716 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:05.312718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:05.312941 (+ 223us) server_negotiation.cc:407] Connection header received
0926 06:57:05.313046 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:05.313047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:05.313056 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:05.313083 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:57:05.313264 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:05.314001 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:05.314718 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:05.315004 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:05.315068 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:05.315238 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:57:05.315268 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:57:05.315313 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:05.315405 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 06:57:05.315427 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260926 06:57:05.315788 23333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:05.312620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45280 (local address 127.0.191.254:40631)
0926 06:57:05.312805 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:05.312808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:05.312816 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:57:05.312858 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:05.312859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:05.312912 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:05.312995 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:57:05.313101 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:05.313730 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:05.314777 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:05.315029 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:05.315258 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:05.315342 (+ 84us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:57:05.315433 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:57:05.315666 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:05.315683 (+ 17us) server_negotiation.cc:299] Negotiation successful
0926 06:57:05.315708 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":44,"threads_started":1}
I20260926 06:57:05.316105 23327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:05.312745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37308 (local address 127.0.191.252:42951)
0926 06:57:05.312777 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:05.312778 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:05.312835 (+ 57us) server_negotiation.cc:407] Connection header received
0926 06:57:05.312919 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:05.312920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:05.312929 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:05.312976 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:57:05.313137 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:05.313891 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:05.314862 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:05.315145 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:05.315233 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:05.315474 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:57:05.315501 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:57:05.315592 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:05.316005 (+ 413us) server_negotiation.cc:299] Negotiation successful
0926 06:57:05.316026 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:57:05.323242 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 22970
I20260926 06:57:05.328320 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 23045
I20260926 06:57:05.332957 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 23120
2026-09-26T06:57:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2780 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:57:05 dist-test-slave-ws74 krb5kdc[23338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:57:05 dist-test-slave-ws74 krb5kdc[23338](info): set up 2 sockets
Sep 26 06:57:05 dist-test-slave-ws74 krb5kdc[23338](info): commencing operation
krb5kdc: starting...
W20260926 06:57:06.357736 767 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/test-user.keytab.
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:57:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:57:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.191.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
Entry for principal kudu/127.0.191.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab.
I20260926 06:57:06.474334 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:44205
--webserver_interface=127.0.191.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:44205,127.0.191.253:33201,127.0.191.252:44553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:57:06.554169 23351 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 06:57:06.554430 23351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:57:06.554458 23351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:57:06.554509 23351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:57:06.555645 23351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:57:06.555713 23351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:57:06.555737 23351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:57:06.555759 23351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:57:06.555789 23351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:57:06.557276 23351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:44205,127.0.191.253:33201,127.0.191.252:44553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kudu_127.0.191.254.keytab
--principal=kudu/127.0.191.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.254:44205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.191.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:57:06.557600 23351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:57:06.557837 23351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:57:06.562690 23359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:06.562692 23356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:06.562727 23357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:57:06.563019 23351 server_base.cc:1034] running on GCE node
I20260926 06:57:06.563268 23351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:57:06.563581 23351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:57:06.564745 23351 hybrid_clock.cc:648] HybridClock initialized: now 1790405826564721 us; error 43 us; skew 500 ppm
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:57:06.567294 23351 init.cc:377] Logged in from keytab as kudu/127.0.191.254@KRBTEST.COM (short username kudu)
I20260926 06:57:06.567770 23351 webserver.cc:460] Webserver started at http://127.0.191.254:42817/ using document root <none> and password file <none>
I20260926 06:57:06.567989 23351 fs_manager.cc:359] Metadata directory not provided
I20260926 06:57:06.568032 23351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:57:06.568121 23351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:57:06.569137 23351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/data/instance:
uuid: "4ad4ca865f3045db9ffdff72d0b5d971"
format_stamp: "Formatted at 2026-09-26 06:57:06 on dist-test-slave-ws74"
server_key: "c3f92fe5725a31e63fdcd8f80d267141"
server_key_iv: "b0689e678aa670a0d93ee721fa78b9c2"
server_key_version: "encryptionkey@0"
I20260926 06:57:06.569516 23351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/wal/instance:
uuid: "4ad4ca865f3045db9ffdff72d0b5d971"
format_stamp: "Formatted at 2026-09-26 06:57:06 on dist-test-slave-ws74"
server_key: "c3f92fe5725a31e63fdcd8f80d267141"
server_key_iv: "b0689e678aa670a0d93ee721fa78b9c2"
server_key_version: "encryptionkey@0"
I20260926 06:57:06.570829 23351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:57:06.571542 23366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:57:06.571731 23351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:57:06.571803 23351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/wal
uuid: "4ad4ca865f3045db9ffdff72d0b5d971"
format_stamp: "Formatted at 2026-09-26 06:57:06 on dist-test-slave-ws74"
server_key: "c3f92fe5725a31e63fdcd8f80d267141"
server_key_iv: "b0689e678aa670a0d93ee721fa78b9c2"
server_key_version: "encryptionkey@0"
I20260926 06:57:06.571856 23351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:57:06.588711 23351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:57:06.590648 23351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:57:06.590761 23351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:57:06.594844 23351 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.254:44205
I20260926 06:57:06.594902 23418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.254:44205 every 8 connection(s)
I20260926 06:57:06.595234 23351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/data/info.pb
I20260926 06:57:06.596200 23419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:57:06.597782 23419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:57:06.599031 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 23351
I20260926 06:57:06.599110 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-0/wal/instance
I20260926 06:57:06.599316 767 external_mini_cluster.cc:1428] Setting key e9d305cf58701bcc15f6f2d2270c5b6b
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.191.253@KRBTEST.COM; defaulting to no policy
I20260926 06:57:06.608639 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.598452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:44205 (local address 127.0.0.1:57724)
0926 06:57:06.598617 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.598658 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:06.599132 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:06.599463 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:06.599465 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:06.599726 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:06.599878 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.599882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.600877 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.600878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.601620 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.601622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.602322 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.602324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.602417 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.602952 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:06.602960 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:06.604740 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:06.607025 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.607032 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.607034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.607239 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.607521 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.607522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.607522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.607669 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.608167 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:06.608172 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:06.608237 (+ 65us) client_negotiation.cc:769] Sending connection context
0926 06:57:06.608319 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 06:57:06.608548 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":64,"thread_start_us":84,"threads_started":1}
I20260926 06:57:06.608645 23422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.598505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57724 (local address 127.0.191.254:44205)
0926 06:57:06.598658 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.598674 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.598752 (+ 78us) server_negotiation.cc:407] Connection header received
0926 06:57:06.599315 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.599317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.599333 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.599427 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.599983 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.600782 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.601703 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.602201 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.602312 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.604843 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.604853 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.604854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.604862 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.606916 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.607336 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.607340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.607340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.607448 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.607756 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.607757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.607758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.608093 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.608172 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.608366 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.608464 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
I20260926 06:57:06.609485 23419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:57:06.609993 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.609845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:33201 (local address 127.0.0.1:34210)
0926 06:57:06.609932 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.609973 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260926 06:57:06.610136 23369 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.191.253:33201: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.253@KRBTEST.COM" created.
I20260926 06:57:06.610592 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.610494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:33201 (local address 127.0.0.1:34216)
0926 06:57:06.610541 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.610569 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 06:57:06.610714 23419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.253:33201: Network error: Client connection negotiation failed: client connection to 127.0.191.253:33201: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kudu_127.0.191.253.keytab.
Entry for principal kudu/127.0.191.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kudu_127.0.191.253.keytab.
I20260926 06:57:06.624296 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.253:33201
--webserver_interface=127.0.191.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kudu_127.0.191.253.keytab
--principal=kudu/127.0.191.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:44205,127.0.191.253:33201,127.0.191.252:44553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:57:06.659976 23419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } attempt: 1
I20260926 06:57:06.660537 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.660375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:33201 (local address 127.0.0.1:34224)
0926 06:57:06.660472 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.660507 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260926 06:57:06.660877 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.660784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:33201 (local address 127.0.0.1:34230)
0926 06:57:06.660828 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.660854 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 06:57:06.660966 23419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.253:33201: Network error: Client connection negotiation failed: client connection to 127.0.191.253:33201: connect: Connection refused (error 111)
W20260926 06:57:06.705338 23426 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 06:57:06.705601 23426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:57:06.705618 23426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:57:06.705639 23426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:57:06.706844 23426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:57:06.706885 23426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:57:06.706899 23426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:57:06.706912 23426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:57:06.706924 23426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:57:06.708441 23426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:44205,127.0.191.253:33201,127.0.191.252:44553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kudu_127.0.191.253.keytab
--principal=kudu/127.0.191.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.253:33201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.191.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:57:06.708707 23426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:57:06.708953 23426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:57:06.713874 23432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:06.713871 23434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:06.713910 23431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:57:06.713910 23426 server_base.cc:1034] running on GCE node
I20260926 06:57:06.714360 23426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:57:06.714643 23426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:57:06.715811 23426 hybrid_clock.cc:648] HybridClock initialized: now 1790405826715800 us; error 40 us; skew 500 ppm
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:57:06.718345 23426 init.cc:377] Logged in from keytab as kudu/127.0.191.253@KRBTEST.COM (short username kudu)
I20260926 06:57:06.718803 23426 webserver.cc:460] Webserver started at http://127.0.191.253:45661/ using document root <none> and password file <none>
I20260926 06:57:06.719033 23426 fs_manager.cc:359] Metadata directory not provided
I20260926 06:57:06.719077 23426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:57:06.719216 23426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:57:06.720352 23426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/data/instance:
uuid: "71d4808e13344a948066bd7ea2b67221"
format_stamp: "Formatted at 2026-09-26 06:57:06 on dist-test-slave-ws74"
server_key: "452f5e2bbcc2d252a01ef7a7580d1397"
server_key_iv: "a37fd1c9ca3036e4d42b1cd514e0bce5"
server_key_version: "encryptionkey@0"
I20260926 06:57:06.720717 23426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/wal/instance:
uuid: "71d4808e13344a948066bd7ea2b67221"
format_stamp: "Formatted at 2026-09-26 06:57:06 on dist-test-slave-ws74"
server_key: "452f5e2bbcc2d252a01ef7a7580d1397"
server_key_iv: "a37fd1c9ca3036e4d42b1cd514e0bce5"
server_key_version: "encryptionkey@0"
I20260926 06:57:06.722134 23426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:57:06.722913 23441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:57:06.723101 23426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:57:06.723203 23426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/wal
uuid: "71d4808e13344a948066bd7ea2b67221"
format_stamp: "Formatted at 2026-09-26 06:57:06 on dist-test-slave-ws74"
server_key: "452f5e2bbcc2d252a01ef7a7580d1397"
server_key_iv: "a37fd1c9ca3036e4d42b1cd514e0bce5"
server_key_version: "encryptionkey@0"
I20260926 06:57:06.723275 23426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:57:06.729121 23419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } attempt: 2
I20260926 06:57:06.729846 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.729604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:33201 (local address 127.0.0.1:34242)
0926 06:57:06.729762 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.729803 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260926 06:57:06.730230 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.730132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:33201 (local address 127.0.0.1:34252)
0926 06:57:06.730178 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.730206 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.253:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 06:57:06.730311 23419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.253:33201: Network error: Client connection negotiation failed: client connection to 127.0.191.253:33201: connect: Connection refused (error 111)
I20260926 06:57:06.737100 23426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:57:06.739251 23426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:57:06.739413 23426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:57:06.743547 23426 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.253:33201
I20260926 06:57:06.743582 23493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.253:33201 every 8 connection(s)
I20260926 06:57:06.743932 23426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/data/info.pb
I20260926 06:57:06.744890 23494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:57:06.746485 23494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:57:06.749343 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 23426
I20260926 06:57:06.749490 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-1/wal/instance
I20260926 06:57:06.749671 767 external_mini_cluster.cc:1428] Setting key 6f05740196e8f8788a34dd8d722739bd
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.253@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:57:06.756158 23422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.747176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.0.191.254:44205)
0926 06:57:06.747254 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.747256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.747404 (+ 148us) server_negotiation.cc:407] Connection header received
0926 06:57:06.747842 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.747843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.747855 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.747920 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.748543 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.749190 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.750387 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.750728 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.750807 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.753259 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.753260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.753261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.753268 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.754762 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.755192 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.755193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.755194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.755253 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.755509 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.755510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.755511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.755630 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.755705 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.755980 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.756065 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 06:57:06.756366 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.747097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:44205 (local address 127.0.0.1:57730)
0926 06:57:06.747250 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.747293 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:06.747745 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:06.748006 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:06.748010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:06.748248 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:06.748444 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.748448 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.749288 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.749289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.750290 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.750293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.750865 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.750866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.751063 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.751698 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:06.751715 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:06.753160 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:06.754871 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.754877 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.754880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.755105 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.755320 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.755321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.755321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.755436 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.755697 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:06.755703 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:06.755844 (+ 141us) client_negotiation.cc:769] Sending connection context
0926 06:57:06.755961 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 06:57:06.756117 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.0.191.252@KRBTEST.COM; defaulting to no policy
I20260926 06:57:06.757097 23494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.191.252@KRBTEST.COM" created.
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.253@KRBTEST.COM for kudu/127.0.191.253@KRBTEST.COM
I20260926 06:57:06.764768 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.757362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:33201 (local address 127.0.0.1:34260)
0926 06:57:06.757449 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.757461 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:06.757501 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:06.757769 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:06.757769 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:06.757883 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:06.757932 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.757933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.758839 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.758840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.759558 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.759560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.760102 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.760130 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.760217 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.760637 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:06.760647 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:06.761866 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:06.763676 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.763677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.763678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.763846 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.764061 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.764062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.764063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.764117 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.764412 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:06.764412 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:06.764458 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 06:57:06.764547 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 06:57:06.764650 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 06:57:06.764859 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.757467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.0.191.253:33201)
0926 06:57:06.757636 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.757640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.757656 (+ 16us) server_negotiation.cc:407] Connection header received
0926 06:57:06.757675 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.757676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.757688 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.757752 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.758005 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.758757 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.759637 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.760009 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.760069 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.761962 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.761970 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.761970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.761977 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.763569 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.763927 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.763937 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.763938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.763995 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.764189 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.764190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.764190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.764335 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.764423 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.764582 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.764722 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260926 06:57:06.765415 23494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:57:06.765787 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.765679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:44553 (local address 127.0.0.1:45184)
0926 06:57:06.765722 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.765765 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 06:57:06.765946 23443 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
I20260926 06:57:06.766378 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.766292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:44553 (local address 127.0.0.1:45200)
0926 06:57:06.766339 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.766362 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 06:57:06.766515 23494 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:44553: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.191.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kudu_127.0.191.252.keytab.
Entry for principal kudu/127.0.191.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kudu_127.0.191.252.keytab.
I20260926 06:57:06.776795 767 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyAhorJ/build/release/bin/kudu
/tmp/dist-test-taskyAhorJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.252:44553
--webserver_interface=127.0.191.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kudu_127.0.191.252.keytab
--principal=kudu/127.0.191.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.191.212:45515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.191.254:44205,127.0.191.253:33201,127.0.191.252:44553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:57:06.815739 23494 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } attempt: 1
I20260926 06:57:06.816283 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.816136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:44553 (local address 127.0.0.1:45206)
0926 06:57:06.816202 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.816245 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 06:57:06.816671 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.816572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:44553 (local address 127.0.0.1:45208)
0926 06:57:06.816618 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.816651 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 06:57:06.816766 23494 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:44553: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
I20260926 06:57:06.821434 23419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } attempt: 3
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for kudu/127.0.191.253@KRBTEST.COM
I20260926 06:57:06.828583 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.821816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.0.191.253:33201)
0926 06:57:06.821875 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.821878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.821889 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:57:06.821936 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.821937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.821950 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.822021 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.822339 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.822831 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.823558 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.823778 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.823855 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.825898 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.825899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.825900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.825920 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.827488 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.827883 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.827884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.827885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.827935 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.828131 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.828131 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.828131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.828231 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.828279 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.828435 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.828523 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 06:57:06.828620 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.821746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:33201 (local address 127.0.0.1:34270)
0926 06:57:06.821792 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.821802 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:06.821874 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:06.822027 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:06.822027 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:06.822194 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:06.822256 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.822258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.822924 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.822925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.823473 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.823475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.823851 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.823852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.823936 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.824404 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:06.824409 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:06.825819 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:06.827567 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.827568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.827568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.827798 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.827999 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.828000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.828000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.828071 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.828289 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:06.828290 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:06.828341 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 06:57:06.828411 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 06:57:06.828519 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 06:57:06.829113 23419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:57:06.829473 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.829339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:44553 (local address 127.0.0.1:45220)
0926 06:57:06.829409 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.829449 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 06:57:06.829795 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.829705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:44553 (local address 127.0.0.1:45224)
0926 06:57:06.829750 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.829777 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 06:57:06.829882 23419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:44553: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
W20260926 06:57:06.859367 23501 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 06:57:06.859594 23501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:57:06.859612 23501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:57:06.859632 23501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:57:06.860785 23501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:57:06.860826 23501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:57:06.860839 23501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:57:06.860852 23501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:57:06.860888 23501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:57:06.861073 23419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } attempt: 1
I20260926 06:57:06.861611 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.861450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:44553 (local address 127.0.0.1:45238)
0926 06:57:06.861523 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.861568 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 06:57:06.862049 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.861942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:44553 (local address 127.0.0.1:45246)
0926 06:57:06.861996 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.862027 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 06:57:06.862157 23419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:44553: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
I20260926 06:57:06.862905 23501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.191.212:45515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.191.254:44205,127.0.191.253:33201,127.0.191.252:44553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/krb5kdc/kudu_127.0.191.252.keytab
--principal=kudu/127.0.191.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.191.252:44553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.191.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 06:04:27 UTC on ccycloud.quasar-krtrcq.root.comops.site
build id 4673739
I20260926 06:57:06.863165 23501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:57:06.863433 23501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:57:06.868700 23509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:06.868716 23506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:57:06.868695 23507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:57:06.868937 23501 server_base.cc:1034] running on GCE node
I20260926 06:57:06.869171 23501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:57:06.869481 23501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:57:06.870668 23501 hybrid_clock.cc:648] HybridClock initialized: now 1790405826870644 us; error 35 us; skew 500 ppm
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:57:06.873242 23501 init.cc:377] Logged in from keytab as kudu/127.0.191.252@KRBTEST.COM (short username kudu)
I20260926 06:57:06.873835 23501 webserver.cc:460] Webserver started at http://127.0.191.252:43423/ using document root <none> and password file <none>
I20260926 06:57:06.874085 23501 fs_manager.cc:359] Metadata directory not provided
I20260926 06:57:06.874154 23501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:57:06.874289 23501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:57:06.875645 23501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/data/instance:
uuid: "db400fcea54e4d0e8869c376981fe190"
format_stamp: "Formatted at 2026-09-26 06:57:06 on dist-test-slave-ws74"
server_key: "12d3b8a42c623aea498854dfa113a23c"
server_key_iv: "b6addcf4107284230af43cb74c0509f8"
server_key_version: "encryptionkey@0"
I20260926 06:57:06.876052 23501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/wal/instance:
uuid: "db400fcea54e4d0e8869c376981fe190"
format_stamp: "Formatted at 2026-09-26 06:57:06 on dist-test-slave-ws74"
server_key: "12d3b8a42c623aea498854dfa113a23c"
server_key_iv: "b6addcf4107284230af43cb74c0509f8"
server_key_version: "encryptionkey@0"
I20260926 06:57:06.877561 23501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:57:06.878427 23516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:57:06.878630 23501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:57:06.878741 23501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/data,/tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/wal
uuid: "db400fcea54e4d0e8869c376981fe190"
format_stamp: "Formatted at 2026-09-26 06:57:06 on dist-test-slave-ws74"
server_key: "12d3b8a42c623aea498854dfa113a23c"
server_key_iv: "b6addcf4107284230af43cb74c0509f8"
server_key_version: "encryptionkey@0"
I20260926 06:57:06.878816 23501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:57:06.884917 23494 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } attempt: 2
I20260926 06:57:06.885324 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.885198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:44553 (local address 127.0.0.1:45248)
0926 06:57:06.885257 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.885296 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 06:57:06.885691 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.885595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:44553 (local address 127.0.0.1:45254)
0926 06:57:06.885640 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.885665 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 06:57:06.885777 23494 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.191.252:44553: Network error: Client connection negotiation failed: client connection to 127.0.191.252:44553: connect: Connection refused (error 111)
I20260926 06:57:06.889467 23501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:57:06.891279 23501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:57:06.891448 23501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:57:06.895303 23501 rpc_server.cc:307] RPC server started. Bound to: 127.0.191.252:44553
I20260926 06:57:06.895332 23568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.191.252:44553 every 8 connection(s)
I20260926 06:57:06.895684 23501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/data/info.pb
I20260926 06:57:06.896601 23569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:57:06.898231 23569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:57:06.901872 767 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyAhorJ/build/release/bin/kudu as pid 23501
I20260926 06:57:06.901993 767 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyAhorJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790405559478510-767-0/minicluster-data/master-2/wal/instance
I20260926 06:57:06.902196 767 external_mini_cluster.cc:1428] Setting key 38f9928e064810c063a27ef58b398816
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.252@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
I20260926 06:57:06.907510 23422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.898933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57732 (local address 127.0.191.254:44205)
0926 06:57:06.898994 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.898996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.899208 (+ 212us) server_negotiation.cc:407] Connection header received
0926 06:57:06.899628 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.899629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.899639 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.899695 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.900270 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.900801 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.901856 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.902170 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.902251 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.904687 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.904689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.904689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.904697 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.906195 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.906577 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.906578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.906579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.906640 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.906886 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.906887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.906887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.907005 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.907080 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.907336 (+ 256us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.907422 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:57:06.907689 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.898913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.254:44205 (local address 127.0.0.1:57732)
0926 06:57:06.899059 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.899099 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:06.899541 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:06.899753 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:06.899756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:06.899999 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:06.900182 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.900185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.900889 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.900890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.901721 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.901723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.902442 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.902444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.902633 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.903216 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:06.903226 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:06.904591 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:06.906270 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.906276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.906278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.906501 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.906709 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.906710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.906710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.906813 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.907071 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:06.907077 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:06.907209 (+ 132us) client_negotiation.cc:769] Sending connection context
0926 06:57:06.907316 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 06:57:06.907459 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.252@KRBTEST.COM
I20260926 06:57:06.908197 23569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:57:06.911454 23573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.902608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45256 (local address 127.0.191.252:44553)
0926 06:57:06.902755 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.902760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.902992 (+ 232us) server_negotiation.cc:407] Connection header received
0926 06:57:06.903074 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.903077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.903100 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.903203 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.903806 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.904521 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.905468 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.905949 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.906022 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.908401 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.908407 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.908408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.908414 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.910208 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.910568 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.910571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.910571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.910634 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.910836 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.910837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.910837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.911010 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.911149 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.911247 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.911331 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.252@KRBTEST.COM for kudu/127.0.191.253@KRBTEST.COM
I20260926 06:57:06.915467 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.908469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.253:33201 (local address 127.0.0.1:34278)
0926 06:57:06.908666 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.908674 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:06.908709 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:06.908846 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:06.908847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:06.909047 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:06.909106 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.909108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.909843 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.909844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.910514 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.910517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.910973 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.910974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.911044 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.911436 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:06.911440 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:06.912718 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:06.914433 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.914434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.914435 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.914645 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.914829 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.914830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.914830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.914896 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.915174 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:06.915175 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:06.915217 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 06:57:06.915290 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 06:57:06.915376 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
I20260926 06:57:06.915503 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.908547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34278 (local address 127.0.191.253:33201)
0926 06:57:06.908589 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.908590 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.908720 (+ 130us) server_negotiation.cc:407] Connection header received
0926 06:57:06.908761 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.908762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.908774 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.908830 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.909180 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.909765 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.910608 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.910905 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.910953 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.912809 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.912810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.912810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.912816 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.914332 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.914717 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.914717 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.914718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.914766 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.914960 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.914960 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.914961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.915086 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.915159 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.915325 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.915403 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:57:06.915877 23569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.254@KRBTEST.COM
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.252@KRBTEST.COM for kudu/127.0.191.252@KRBTEST.COM
I20260926 06:57:06.920046 23422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.912215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57748 (local address 127.0.191.254:44205)
0926 06:57:06.912260 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.912261 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.912270 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:57:06.912289 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.912290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.912298 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.912364 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.912860 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.913566 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.914572 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.914867 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.914943 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.917480 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.917482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.917482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.917489 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.918976 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.919271 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.919272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.919272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.919332 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.919533 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.919533 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.919534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.919667 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.919822 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.919860 (+ 38us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.919935 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 06:57:06.922571 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.916122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:44553 (local address 127.0.0.1:45260)
0926 06:57:06.916214 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.916225 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:06.916268 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:06.916376 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:06.916376 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:06.916500 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:06.916525 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.916526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.917145 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.917145 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.917810 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.917812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.918169 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.918169 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.918217 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.918587 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:06.918591 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:06.919847 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:06.921636 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.921637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.921638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.921808 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.922004 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.922004 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.922004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.922051 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.922274 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:06.922274 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:06.922303 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 06:57:06.922378 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:57:06.922466 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 06:57:06.922613 23573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.916179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45260 (local address 127.0.191.252:44553)
0926 06:57:06.916213 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.916215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.916262 (+ 47us) server_negotiation.cc:407] Connection header received
0926 06:57:06.916305 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.916306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.916317 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.916374 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.916598 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.917047 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.917883 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.918088 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.918169 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.919935 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.919936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.919936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.919943 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.921565 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.921874 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.921875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.921875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.921939 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.922107 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.922108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.922108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.922220 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.922283 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.922409 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.922510 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":17}
I20260926 06:57:06.924059 23569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190: Bootstrap starting.
I20260926 06:57:06.924855 23569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190: Neither blocks nor log segments found. Creating new log.
I20260926 06:57:06.925139 23569 log.cc:824] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190: Log is configured to *not* fsync() on all Append() calls
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.191.253@KRBTEST.COM
I20260926 06:57:06.925971 23569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190: No bootstrap required, opened a new log
I20260926 06:57:06.927999 23569 raft_consensus.cc:348] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } }
I20260926 06:57:06.928161 23569 raft_consensus.cc:729] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db400fcea54e4d0e8869c376981fe190, State: Initialized, Role: FOLLOWER
I20260926 06:57:06.928287 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.920919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34282 (local address 127.0.191.253:33201)
0926 06:57:06.920955 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.920957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.921016 (+ 59us) server_negotiation.cc:407] Connection header received
0926 06:57:06.921037 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.921038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.921049 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.921121 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.921658 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.922301 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.923103 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.923424 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.923494 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.925715 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.925717 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.925717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.925723 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.927211 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.927571 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.927572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.927572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.927617 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.927812 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.927813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.927813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.927912 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.928017 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.928119 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.928170 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:57:06.928265 23569 consensus_queue.cc:260] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } }
I20260926 06:57:06.928602 23569 sys_catalog.cc:564] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:57:06.928697 23575 sys_catalog.cc:455] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } } }
I20260926 06:57:06.928802 23575 sys_catalog.cc:458] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:57:06.930004 767 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260926 06:57:06.930437 23586 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:57:06.930493 23586 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:57:06.930994 23393 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4ad4ca865f3045db9ffdff72d0b5d971"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57748
I20260926 06:57:06.932250 23501 master_runner.cc:186] Error getting master registration for 127.0.191.254:44205: OK, instance_id { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" instance_seqno: 1790405826593674 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 06:57:06.933957 23501 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 06:57:06.937276 23419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } attempt: 2
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.254@KRBTEST.COM for kudu/127.0.191.252@KRBTEST.COM
I20260926 06:57:06.945562 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.937652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:44553 (local address 127.0.0.1:45272)
0926 06:57:06.937702 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.937715 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:06.937764 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:06.937912 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:06.937913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:06.938093 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:06.938140 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.938142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.939059 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.939061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.939889 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.939892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.940366 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.940367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.940440 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.940995 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:06.941001 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:06.942503 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:06.944386 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.944388 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.944388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.944622 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.944828 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.944829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.944829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.944894 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.945208 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:06.945209 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:06.945256 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 06:57:06.945328 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:57:06.945459 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 06:57:06.945572 23573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.937725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45272 (local address 127.0.191.252:44553)
0926 06:57:06.937786 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.937788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.937812 (+ 24us) server_negotiation.cc:407] Connection header received
0926 06:57:06.937835 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.937836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.937847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.937898 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.938221 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.938969 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.939977 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.940276 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.940357 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.942605 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.942606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.942607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.942621 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.944294 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.944698 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.944700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.944700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.944758 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.944958 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.944959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.944959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.945132 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.945206 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.945391 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.945471 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 06:57:06.947353 23419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971: Bootstrap starting.
I20260926 06:57:06.948100 23419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971: Neither blocks nor log segments found. Creating new log.
I20260926 06:57:06.948410 23419 log.cc:824] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971: Log is configured to *not* fsync() on all Append() calls
I20260926 06:57:06.949287 23419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971: No bootstrap required, opened a new log
I20260926 06:57:06.950670 23419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } }
I20260926 06:57:06.950814 23419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ad4ca865f3045db9ffdff72d0b5d971, State: Initialized, Role: FOLLOWER
I20260926 06:57:06.950902 23419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } }
I20260926 06:57:06.951172 23589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } } }
I20260926 06:57:06.951251 23589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:57:06.951251 23419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:57:06.951392 23393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 06:57:06.951506 23393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:57:06.952292 23393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } }
I20260926 06:57:06.952879 23543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "db400fcea54e4d0e8869c376981fe190"
I20260926 06:57:06.953135 23543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190 [term 0 FOLLOWER]: Advancing to term 1
W20260926 06:57:06.953608 23600 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:57:06.953666 23600 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:57:06.954185 23543 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ad4ca865f3045db9ffdff72d0b5d971 in term 1.
I20260926 06:57:06.954559 23368 leader_election.cc:302] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ad4ca865f3045db9ffdff72d0b5d971, db400fcea54e4d0e8869c376981fe190; no voters:
I20260926 06:57:06.954731 23589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:57:06.954864 23589 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [term 1 LEADER]: Becoming Leader. State: Replica: 4ad4ca865f3045db9ffdff72d0b5d971, State: Running, Role: LEADER
I20260926 06:57:06.954941 23589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } }
I20260926 06:57:06.954981 23393 leader_election.cc:288] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [CANDIDATE]: Term 1 election: Requested vote from peers 71d4808e13344a948066bd7ea2b67221 (127.0.191.253:33201), db400fcea54e4d0e8869c376981fe190 (127.0.191.252:44553)
I20260926 06:57:06.955425 23589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ad4ca865f3045db9ffdff72d0b5d971. Latest consensus state: current_term: 1 leader_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } } }
I20260926 06:57:06.955492 23589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [sys.catalog]: This master's current role is: LEADER
I20260926 06:57:06.957427 23351 master_runner.cc:186] Error getting master registration for 127.0.191.253:33201: OK, instance_id { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" instance_seqno: 1790405826742380 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 06:57:06.959772 23351 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 06:57:06.961560 23604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:57:06.961786 23604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:57:06.963994 23543 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190 [term 1 FOLLOWER]: Refusing update from remote peer 4ad4ca865f3045db9ffdff72d0b5d971: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:57:06.964825 23589 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:57:06.966279 23575 sys_catalog.cc:455] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ad4ca865f3045db9ffdff72d0b5d971. Latest consensus state: current_term: 1 leader_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } } }
I20260926 06:57:06.966383 23575 sys_catalog.cc:458] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:57:06.972311 23422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.956656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57754 (local address 127.0.191.254:44205)
0926 06:57:06.956698 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.956700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.962722 (+ 6022us) server_negotiation.cc:407] Connection header received
0926 06:57:06.962835 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.962836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.962848 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.962881 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.963784 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.964615 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.965814 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.966135 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.966181 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.969095 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.969096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.969097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.969104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.970959 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.971421 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.971422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.971423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.971485 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.971700 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.971701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.971701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.971832 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.971908 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.972127 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.972211 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:57:06.976974 23494 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } attempt: 3
I20260926 06:57:06.979044 23612 mvcc.cc:205] Tried to move back new op lower bound from 7333502267240402944 to 7333502267208105984. Current Snapshot: MvccSnapshot[applied={T|T < 7333502267240402944}]
I20260926 06:57:06.979278 23589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } } }
I20260926 06:57:06.979382 23589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [sys.catalog]: This master's current role is: LEADER
I20260926 06:57:06.979598 23575 sys_catalog.cc:455] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } } }
I20260926 06:57:06.979698 23575 sys_catalog.cc:458] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:57:06.979928 23613 mvcc.cc:205] Tried to move back new op lower bound from 7333502267240402944 to 7333502267208105984. Current Snapshot: MvccSnapshot[applied={T|T < 7333502267240402944}]
I20260926 06:57:06.980144 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.957173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34292 (local address 127.0.191.253:33201)
0926 06:57:06.957217 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.957219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.967072 (+ 9853us) server_negotiation.cc:407] Connection header received
0926 06:57:06.967480 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.967482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.967492 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.967525 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.968198 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.969009 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.973902 (+ 4893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.974295 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.974352 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.976102 (+ 1750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.976104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.976104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.976113 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.978741 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.979176 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.979177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.979177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.979240 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.979475 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.979476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.979476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.979610 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.979704 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.979937 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.980037 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:57:06.980896 23604 catalog_manager.cc:1347] Generated new cluster ID: 80d1c83a0d804c1e83fdb2aae4b7a741
I20260926 06:57:06.980954 23604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:57:06 dist-test-slave-ws74 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790405826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.191.253@KRBTEST.COM for kudu/127.0.191.252@KRBTEST.COM
I20260926 06:57:06.990381 23604 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:57:06.992534 23573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.956845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.0.191.252:44553)
0926 06:57:06.956900 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.956902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.966902 (+ 10000us) server_negotiation.cc:407] Connection header received
0926 06:57:06.967124 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.967125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.967139 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.967171 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.970723 (+ 3552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.971495 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.973517 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.973853 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.973895 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.976691 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.976693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.976693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.976701 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.978785 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.980669 (+ 1884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.980670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.980671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.980737 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.991784 (+ 11047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.991785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.991786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.991939 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.992039 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.992326 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.992427 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 06:57:06.992810 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.977250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.191.252:44553 (local address 127.0.0.1:45292)
0926 06:57:06.977397 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 06:57:06.977410 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:57:06.977452 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:57:06.981092 (+ 3640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:57:06.981093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:57:06.981248 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:57:06.981292 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.981294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.982943 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.982944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.983753 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:57:06.983755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.984618 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.984619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:57:06.984693 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.985219 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:57:06.985224 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:57:06.986765 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:57:06.991460 (+ 4695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.991461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.991462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.991681 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.992195 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:57:06.992196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:57:06.992196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:57:06.992256 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:57:06.992519 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:57:06.992519 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:57:06.992563 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 06:57:06.992609 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 06:57:06.992708 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260926 06:57:06.994242 23615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:06.977529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45292 (local address 127.0.191.252:44553)
0926 06:57:06.980967 (+ 3438us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:06.980969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:06.980977 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:57:06.980997 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:06.980998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:06.981010 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:06.981077 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:06.981428 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.982800 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.983917 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:06.984508 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:06.986875 (+ 2367us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:06.986993 (+ 118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:57:06.986994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:57:06.986994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:57:06.987000 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:57:06.991331 (+ 4331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.992065 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.992066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.992067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.992116 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:57:06.992325 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:57:06.992326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:57:06.992326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:57:06.992445 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:57:06.993784 (+ 1339us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:06.993833 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 06:57:06.993926 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3374,"thread_start_us":103,"threads_started":1}
I20260926 06:57:06.995008 23604 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:57:06.996688 23494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221: Bootstrap starting.
I20260926 06:57:06.997546 23494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221: Neither blocks nor log segments found. Creating new log.
I20260926 06:57:06.997879 23494 log.cc:824] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221: Log is configured to *not* fsync() on all Append() calls
I20260926 06:57:07.003288 23604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971: Generated new TSK 0
I20260926 06:57:07.003546 23604 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:57:07.006125 23494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221: No bootstrap required, opened a new log
I20260926 06:57:07.008374 23494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } }
I20260926 06:57:07.008531 23494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71d4808e13344a948066bd7ea2b67221, State: Initialized, Role: FOLLOWER
I20260926 06:57:07.008615 23494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } }
I20260926 06:57:07.008977 23494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:57:07.008934 23620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } } }
I20260926 06:57:07.009130 23620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:57:07.009426 23468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "71d4808e13344a948066bd7ea2b67221"
I20260926 06:57:07.009523 23467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:57:07.010478 23467 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221 [term 1 FOLLOWER]: Refusing update from remote peer 4ad4ca865f3045db9ffdff72d0b5d971: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:57:07.010895 23602 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:57:07.011776 23468 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ad4ca865f3045db9ffdff72d0b5d971 in term 1.
I20260926 06:57:07.015225 23620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ad4ca865f3045db9ffdff72d0b5d971. Latest consensus state: current_term: 1 leader_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } } }
I20260926 06:57:07.015400 23620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:57:07.015486 23620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } } }
I20260926 06:57:07.015897 23620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:57:07.018599 23629 mvcc.cc:205] Tried to move back new op lower bound from 7333502267399057408 to 7333502267208105984. Current Snapshot: MvccSnapshot[applied={T|T < 7333502267240402944}]
W20260926 06:57:07.020509 23635 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:57:07.020619 23635 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 06:57:07.036901 23498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:07.035133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34296 (local address 127.0.191.253:33201)
0926 06:57:07.035180 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:07.035182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:07.036163 (+ 981us) server_negotiation.cc:407] Connection header received
0926 06:57:07.036295 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:07.036296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:07.036308 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:07.036343 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:07.036829 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34296: BlockingRecv error: recv got EOF from 127.0.0.1:34296 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260926 06:57:07.037904 23422 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:07.035560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.0.191.254:44205)
0926 06:57:07.035602 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:07.035604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:07.037335 (+ 1731us) server_negotiation.cc:407] Connection header received
0926 06:57:07.037477 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:07.037478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:07.037489 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:07.037521 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:07.037839 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57760: BlockingRecv error: recv got EOF from 127.0.0.1:57760 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260926 06:57:07.038802 23615 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:07.035896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45294 (local address 127.0.191.252:44553)
0926 06:57:07.035946 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:07.035948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:07.038263 (+ 2315us) server_negotiation.cc:407] Connection header received
0926 06:57:07.038383 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:07.038384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:07.038396 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:07.038428 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:07.038737 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45294: BlockingRecv error: recv got EOF from 127.0.0.1:45294 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260926 06:57:07.041272 23498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:07.040496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34300 (local address 127.0.191.253:33201)
0926 06:57:07.040532 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:07.040533 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:07.040738 (+ 205us) server_negotiation.cc:407] Connection header received
0926 06:57:07.040848 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:07.040849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:07.040869 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:07.040902 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:07.041223 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34300: BlockingRecv error: recv got EOF from 127.0.0.1:34300 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 06:57:07.044486 23498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:07.043701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34312 (local address 127.0.191.253:33201)
0926 06:57:07.043735 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:07.043736 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:07.043982 (+ 246us) server_negotiation.cc:407] Connection header received
0926 06:57:07.044081 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:07.044082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:07.044092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:07.044124 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:07.044445 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34312: BlockingRecv error: recv got EOF from 127.0.0.1:34312 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 06:57:07.048432 23498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:07.047649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34322 (local address 127.0.191.253:33201)
0926 06:57:07.047683 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:07.047685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:07.047919 (+ 234us) server_negotiation.cc:407] Connection header received
0926 06:57:07.048020 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:07.048020 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:07.048030 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:07.048062 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:07.048386 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34322: BlockingRecv error: recv got EOF from 127.0.0.1:34322 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 06:57:07.054452 23498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:07.053638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34330 (local address 127.0.191.253:33201)
0926 06:57:07.053673 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:07.053674 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:07.053922 (+ 248us) server_negotiation.cc:407] Connection header received
0926 06:57:07.054024 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:07.054025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:07.054035 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:07.054067 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:07.054405 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34330: BlockingRecv error: recv got EOF from 127.0.0.1:34330 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 06:57:07.065150 23498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:07.063882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34332 (local address 127.0.191.253:33201)
0926 06:57:07.063929 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:07.063931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:07.064253 (+ 322us) server_negotiation.cc:407] Connection header received
0926 06:57:07.064389 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:07.064390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:07.064404 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:07.064439 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:07.065087 (+ 648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34332: BlockingRecv error: recv got EOF from 127.0.0.1:34332 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260926 06:57:07.084117 23498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:07.083025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34348 (local address 127.0.191.253:33201)
0926 06:57:07.083074 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:07.083076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:07.083439 (+ 363us) server_negotiation.cc:407] Connection header received
0926 06:57:07.083563 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:07.083565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:07.083578 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:07.083614 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:07.084051 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34348: BlockingRecv error: recv got EOF from 127.0.0.1:34348 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260926 06:57:07.119040 23498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:07.117957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34362 (local address 127.0.191.253:33201)
0926 06:57:07.118010 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:07.118013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:07.118343 (+ 330us) server_negotiation.cc:407] Connection header received
0926 06:57:07.118482 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:07.118483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:07.118497 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:07.118532 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:07.118974 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34362: BlockingRecv error: recv got EOF from 127.0.0.1:34362 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260926 06:57:07.185693 23498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:07.184714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.0.191.253:33201)
0926 06:57:07.184768 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:07.184770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:07.184945 (+ 175us) server_negotiation.cc:407] Connection header received
0926 06:57:07.185033 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:07.185034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:07.185044 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:07.185098 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:07.185604 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34370: BlockingRecv error: recv got EOF from 127.0.0.1:34370 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260926 06:57:07.315747 23498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:07.314900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34376 (local address 127.0.191.253:33201)
0926 06:57:07.314998 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:07.315001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:07.315102 (+ 101us) server_negotiation.cc:407] Connection header received
0926 06:57:07.315206 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:07.315207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:07.315219 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:07.315296 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:07.315670 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34376: BlockingRecv error: recv got EOF from 127.0.0.1:34376 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 06:57:07.521847 23693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4ca865f3045db9ffdff72d0b5d971" member_type: VOTER last_known_addr { host: "127.0.191.254" port: 44205 } } peers { permanent_uuid: "71d4808e13344a948066bd7ea2b67221" member_type: VOTER last_known_addr { host: "127.0.191.253" port: 33201 } } peers { permanent_uuid: "db400fcea54e4d0e8869c376981fe190" member_type: VOTER last_known_addr { host: "127.0.191.252" port: 44553 } } }
I20260926 06:57:07.522001 23693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ad4ca865f3045db9ffdff72d0b5d971 [sys.catalog]: This master's current role is: LEADER
W20260926 06:57:07.573808 23498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:57:07.573025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34392 (local address 127.0.191.253:33201)
0926 06:57:07.573097 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:07.573099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:07.573125 (+ 26us) server_negotiation.cc:407] Connection header received
0926 06:57:07.573215 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:07.573216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:07.573234 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:07.573300 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:57:07.573735 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34392: BlockingRecv error: recv got EOF from 127.0.0.1:34392 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 06:57:07.930922 23586 catalog_manager.cc:1259] Loaded cluster ID: 80d1c83a0d804c1e83fdb2aae4b7a741
I20260926 06:57:07.931002 23586 catalog_manager.cc:1552] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190: loading cluster ID for follower catalog manager: success
I20260926 06:57:07.931824 23586 catalog_manager.cc:1574] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190: acquiring CA information for follower catalog manager: success
I20260926 06:57:07.932142 23586 catalog_manager.cc:1602] T 00000000000000000000000000000000 P db400fcea54e4d0e8869c376981fe190: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:57:08.021100 23635 catalog_manager.cc:1259] Loaded cluster ID: 80d1c83a0d804c1e83fdb2aae4b7a741
I20260926 06:57:08.021180 23635 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221: loading cluster ID for follower catalog manager: success
I20260926 06:57:08.022199 23635 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221: acquiring CA information for follower catalog manager: success
I20260926 06:57:08.022547 23635 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 71d4808e13344a948066bd7ea2b67221: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:57:08.089242 23706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:08.086889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.0.191.253:33201)
0926 06:57:08.086998 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:08.087000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:08.087011 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:57:08.087063 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:08.087064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:08.087112 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:08.087179 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:57:08.087339 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:08.087830 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:08.088554 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:08.088739 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:08.088788 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:08.088942 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:57:08.089059 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:57:08.089118 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:08.089158 (+ 40us) server_negotiation.cc:299] Negotiation successful
0926 06:57:08.089175 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":53,"threads_started":1}
I20260926 06:57:08.093634 23712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:08.091094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45300 (local address 127.0.191.252:44553)
0926 06:57:08.091212 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:08.091214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:08.091223 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:57:08.091266 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:08.091267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:08.091300 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:08.091379 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:57:08.091567 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:08.092060 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:08.092786 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:08.092986 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:08.093040 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:08.093163 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:57:08.093259 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:57:08.093314 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:08.093485 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 06:57:08.093509 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":75,"threads_started":1}
I20260926 06:57:08.098008 23706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:08.095239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.0.191.253:33201)
0926 06:57:08.095278 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:08.095280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:08.095384 (+ 104us) server_negotiation.cc:407] Connection header received
0926 06:57:08.095484 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:08.095486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:08.095497 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:08.095549 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:57:08.095634 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:08.096394 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:08.097239 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:08.097516 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:08.097590 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:08.097760 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:57:08.097788 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:57:08.097830 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:08.097884 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 06:57:08.097930 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:57:08.098496 23712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:08.095442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45302 (local address 127.0.191.252:44553)
0926 06:57:08.095478 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:08.095480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:08.095675 (+ 195us) server_negotiation.cc:407] Connection header received
0926 06:57:08.095778 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:08.095779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:08.095789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:08.095836 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:57:08.095943 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:08.096382 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:08.097574 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:08.097872 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:08.097943 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:08.098081 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:57:08.098111 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:57:08.098138 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:08.098375 (+ 237us) server_negotiation.cc:299] Negotiation successful
0926 06:57:08.098397 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:57:08.099026 23719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:57:08.095212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57770 (local address 127.0.191.254:44205)
0926 06:57:08.095308 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0926 06:57:08.095310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:57:08.095385 (+ 75us) server_negotiation.cc:407] Connection header received
0926 06:57:08.095553 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:57:08.095554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:57:08.095610 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:57:08.095699 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:57:08.095817 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:08.096547 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:08.097984 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:57:08.098266 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:57:08.098325 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:57:08.098543 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:57:08.098647 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:57:08.098702 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 06:57:08.098781 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 06:57:08.098809 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":44,"threads_started":1}
I20260926 06:57:08.101087 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 23351
I20260926 06:57:08.106480 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 23426
I20260926 06:57:08.111615 767 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyAhorJ/build/release/bin/kudu with pid 23501
2026-09-26T06:57:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2779 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5559 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (268640 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST